Compare commits

...
Author SHA1 Message Date
Daniel Hiltgen db9c11707b win: fix dependency gathering 2026-05-07 15:59:48 -07:00
Daniel Hiltgen 6b93c29ca1 ci: fix windows dependencies 2026-05-07 11:32:52 -07:00
Daniel Hiltgen cb98d5966e ci: windows mlx tuning
Shorten long-tail on build, and get OllamaSetup.exe back under 2g limit
2026-05-07 07:35:22 -07:00
Daniel Hiltgen 0b45854537 ci: fix windows rocm build 2026-05-06 20:01:09 -07:00
Daniel Hiltgen 140fcf2bd0 ci: fix windows llama-server build 2026-05-06 19:38:05 -07:00
Daniel Hiltgen 9c925a2d29 ci: fix windows MLX build 2026-05-06 19:20:48 -07:00
Daniel Hiltgen d68fd01fa6 refine implementation 2026-05-06 17:26:05 -07:00
jmorganca 868b63ff77 llama/compat: load Ollama-format GGUFs in llama-server
Squashed from upstream/jmorganca/llama-compat on 2026-04-29.
Source tip: 0c33775d37.

Original source commits:
- 25223160d llama/compat: add in-memory shim so llama-server can load Ollama-format GGUFs
- 7449b539a llm,server: route Ollama-format gemma3 blobs through llama/compat
- 436f2e2b1 llama/compat: make patch-apply idempotent
- 8c2c9d4c8 llama/compat: extend gemma3 handler to cover 1B and 270M blobs
- 021389f7b llama/compat: shrink clip.cpp injection from 18 lines to 1
- 61b367ec2 llama/compat: shrink patch to pure call-site hooks (34 -> 20 lines)
- 36049361c llama/compat: simplify shim (gemma3-tested)
- 8fa664865 llama/compat: add qwen35moe text handler
- db0c74530 llama/compat: add qwen35moe vision (clip) support
- 2a388da77 llama/compat: split shared infra into a util TU
- 9a69a17dc llama/compat: document non-public API dependencies
- d0f38a915 llama/compat: add gpt-oss and lfm2 handlers
- 086071822 llama/compat: add mistral3 text handler (vision TODO)
- 63bde9ff7 llama/compat: add mistral3 vision (clip) support
- 3a57b89d5 llama/compat: apply LLaMA RoPE permute to mistral3 vision Q/K
- 99cb87439 llama/compat: add qwen35, gemma4, deepseek-ocr handlers
- 2c7850dba llama/compat: add nemotron_h_moe handler (latent FFN + MTP skip)
- 9e3b54225 llama/compat: add llama4 text + clip handlers
- 034fee349 llama/compat: add gemma4 clip handler (gemma4v projector)
- 9945c5a93 server: remove dhiltgen/* compat redirect table
- 5d4539101 llama/compat: rewrite gemma4 tokenizer model to BPE
- 7e0765327 llama/compat: add glm-ocr text handler + text-loader load-op hook
- f1bd1a25a llama/compat: add glm-ocr clip handler (glm4v projector)
- 4b5cf3420 llama/compat: collapse text-loader hook back to one new patch line
- eb4ecf4fc llama/compat: extend gemma4 clip handler to gemma4a (audio)
- a23a5e76f llama/compat: fix gemma4a per-block norm tensor mapping
- cd2dcaff4 llama/compat: add embeddinggemma handler
- 1ce8a6b26 llama/compat: add qwen3-vl + qwen2.5-vl handlers
- fd98ffa1e llama/compat: add gemma3n + glm4moelite handlers
- cc7bdf0bc llama/compat: handle null buft in maybe_load_tensor
- 0c33775d3 llama/compat: disable mmap when load_op transforms text-side tensors
2026-05-06 17:26:05 -07:00
Daniel Hiltgen 31e336791a runner: Remove CGO engines, use llama-server exclusively for GGML models
Remove the vendored GGML and llama.cpp backend, CGO runner, Go model
implementations, and sample.  llama-server (built from upstream llama.cpp via
FetchContent) is now the sole inference engine for GGUF-based models.
(Safetensor based models continue to run on the new MLX engine.)  This allows
us to more rapidly pick up new capabilities and fixes from llama.cpp as they
come out.

On windows this now requires recent AMD driver versions to support ROCm v7 as
llama.cpp currently does not support building against v6.
2026-05-06 17:26:05 -07:00
Daniel Hiltgen 94c40c9ad1 broad lint fixes to sidestep CI scope glitch 2026-05-06 17:26:05 -07:00
Parth Sareen bab59072fb launch: add plan-aware model gating (#16027) 2026-05-06 14:34:26 -07:00
Eva H 7c2c36bda2 cmd/launch: improve integration backup UX (#15907) 2026-05-06 11:32:54 -04:00
Parth Sareen d319227df0 server: cache show responses (#15967) 2026-05-05 14:40:18 -07:00
Daniel Hiltgen 2d84ec939c mlx: partial cleanup of imagegen layout (#15435)
* mlx: partial cleanup of imagegen layout

This moves part of the imagegen safetensors code to the new package.

* test: remove flaky timing test
2026-05-05 14:15:30 -07:00
Patrick DevineandDaniel Hiltgen 15e6076d79 mlx: Gemma4 MTP speculative decoding (#15980)
This change adds support for MTP (multi-token prediction) speculative decoding for the
gemma4 model family.

It includes:
  * support for importing safetensors based gemma4 draft models with `ollama create`
  * a new DRAFT command in the Modelfile for specifying draft models
  * a --quantize-draft flag for the ollama create command to quantize the draft model
  * cache support for speculation
  * changes to the rotating cache to be able to handle MTP correctly
  * sampling support for draft model token prediction

---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
2026-05-05 08:55:04 -07:00
Parth Sareen 4017af96cd go: bump to 1.26 (#15904) 2026-05-03 23:24:35 -07:00
Daniel Hiltgen 534342e7e2 Update MLX and MLX-C with threading fixes (#15845)
* Update MLX and MLX-C

* Run MLX CGO work on a locked OS thread

MLX now relies on OS-thread-local execution state for streams, encoders, and caches. Add an mlxthread executor backed by runtime.LockOSThread and route runner initialization, model load, inference, status memory reads, and cleanup through the worker so Go goroutine migration cannot split MLX state across native threads.

Also stop caching default MLX streams before the runner owns the thread and add worker/threaded MLX regression tests.

* mlx: use common status writer

* mlx: bundle missing libjaccl on arm64

Inspired by #15793

* review comments
2026-05-03 10:03:14 -07:00
Parth Sareen 9ba5a04914 launch: claude app (#15937) 2026-05-02 19:19:57 -07:00
Bruce MacDonald 938ca6e274 app: source featured models from experimental recommendations endpoint (#15909)
Replace the hardcoded FEATURED_MODELS list with the
/api/experimental/model-recommendations endpoint so the picker stays in
sync with server-driven recommendations. Inline the merge into useModels
(recommendations first, then the rest of /api/tags) and drop the
standalone mergeModels util.
2026-05-01 11:10:20 -07:00
Pratham Agarwal 8f39fff70b fix: resolve OpenClaw gateway launch timeout on Windows by enforcing IPv4 loopback (#15726) 2026-04-30 22:20:08 -04:00
Daniel Hiltgen 4fe5609563 metal: harden for ggml initialization failures (#15755)
* metal: harden for ggml initialization failures

ggml_metal_device_init performs a probe to verify the tensor API compiles.  On
some systems this passes, even though kernel coverage isn't complete, which
results in a later crash when compiling the real kernels.  This change adds a
single retry if any of the error strings match this failure mode to disable the
tensor API.  It also hardens an error case in the Go initDevices to detect
device initialization failures and panic instead of crashing later on a nil
array entry.

Fixes #15734

* review comments

* review comments
2026-04-30 16:28:03 -07:00
Bruce MacDonald 917324bb4d app: remove ollama update url env var used for testing (#15905) 2026-04-30 13:14:08 -07:00
Parth Sareen c7c2837c96 renderers: update gemma4 renderer (#15886) 2026-04-29 18:40:23 -07:00
Parth Sareen b6447caebc launch: use vram bytes for model recommendations (#15885) 2026-04-29 18:40:14 -07:00
Eva H bad32c7244 launch/docs: fix title for pool (#15883) 2026-04-29 17:18:44 -04:00
Eva H ab2e005bf7 app: align the app launch page with ollama launch (#15753) 2026-04-29 14:45:19 -04:00
Parth Sareen 321cc8a2ba server/launch: add model recommendations cache endpoint (#15868) 2026-04-28 17:09:04 -07:00
Daniel HiltgenandEva Ho 87288ced4f New models (#15861)
* mlx: add laguna model support

* convert: support fp8 safetensors import

Decode HF F8_E4M3 safetensors with block scale companions into GGUF-supported tensor types, and record which output tensors came from FP8 source weights.

Use that source-precision metadata during create quantization: default FP8-sourced GGUFs to Q8_0, keep non-FP8 tensors at their original precision for Q8_0, and promote non-FP8 quantizable tensors to Q8_0 for Q4_K requests.

* ggml: add laguna model support

* server: preserve generate logprobs with builtin parsers

Generate requests were dropping logprob-only chunks whenever a builtin parser buffered visible content. Chat already handled this case, but generate only forwarded chunks with visible response, thinking, or tool-call output.

Keep generate chunks that carry logprobs even when the builtin parser has not flushed visible content yet, and add a regression test that exercises the behavior with a generic thinking parser.

* review comments - perf improvements

* ggml: implement nemotron 3 nano omni

* add poolside integration

* update poolside doc

* adapt to new cache setup

* fix test

* fix test

---------

Co-authored-by: Eva Ho <hoyyeva@gmail.com>
2026-04-28 11:50:12 -07:00
Jesse Gross 2bbe2405fe mlxrunner: decouple models from attention cache storage layout
Models build their own attention masks and read K/V directly from
the cache's buffers, which ties them to the cache's storage layout.
That blocks multi-sequence batching — right-padded rows need a
query-padding mask composed onto every model — and rules out
variants like paged attention where K/V isn't one contiguous tensor.

Caches now hand back a per-layer KVHistory holding post-update K, V,
and a MaskApplier that merges the cache's storage restrictions into
the model's logical mask. Models describe their mask in logical
terms; SDPA composes model, padding, and applier contributions and
dispatches to the kernel's causal or no-mask fast path when it can.
KVHistory still exposes K, V, and the composed mask for manual
attention paths (e.g. CUDA prefill at head_dim > 128).

Performance for single-sequence inference is unchanged.
2026-04-27 20:04:46 -07:00
Jesse Gross bd21678b16 mlxrunner: apply RoPE at per-row positions
Switch RoPE from the scalar-offset kernel (mlx_fast_rope) to the
array-offset one (mlx_fast_rope_dynamic) so each batch row can start
at its own position. The pipeline tracks the current position locally
and passes it to the model through Batch.SeqOffsets; each model
materializes that slice into an int32 array for the RoPE call.

Single-sequence behavior is unchanged; this is the wiring needed
before the runner can batch independent sequences.
2026-04-27 20:04:46 -07:00
Jesse Gross 088dfd89a8 mlxrunner: wrap model forward inputs in a Batch struct
Gives a single extension point for per-call context (positions,
sequence IDs, masks) as multi-sequence batching grows, without having
to churn every model's Forward signature again.
2026-04-27 20:04:46 -07:00
Eva H 3cab8a7b02 app/server: fix desktop app startup killing active ollama launch sessions (#15657) 2026-04-27 22:52:53 -04:00
Daniel Hiltgen 03aee88186 mlx: Support NVIDIA TensorRT Model Optimizer import (#15566)
* mlx: Support NVIDIA TensorRT Model Optimizer import

* x/create: support FP8 safetensors import

Decode HF F8_E4M3 safetensors with block scale companions into MLX-importable tensor blobs, including compressed-tensors weight_scale metadata, packed NVFP4 layouts, and mixed-precision tensor headers.

Use that source-precision metadata during create quantization: default FP8-sourced imports to mxfp8, allow source FP8 to target MLX low-bit formats, preserve source-quantized NVFP4 layouts, selectively keep or promote tensors based on their source precision, and detect quantized dtype from mixed-precision safetensors manifests.

* review comments
2026-04-27 18:28:10 -07:00
Daniel Hiltgen ec9b4e9e47 tokenizer: fix multi-regex BPE offset handling (#15844)
Use the current fragment offset when emitting unmatched spans during multi-regex BPE splitting. This avoids duplicating earlier prompt text and inflating token counts for multi-stage BPE tokenizers.
2026-04-27 14:14:27 -07:00
Jesse Gross 4656a07e56 mlxrunner: batch the sampler across multiple sequences
Register sequences with Add/Remove; each Sample call takes any subset of
registered slots and samples one token per row, appending to each slot's
ring-buffer history. When all slots share Options and penalty rings are
full, one fused transform pass runs over the whole batch via a persistent
pooled history tensor; otherwise calls fall back to per-slot serial
processing indexed against the same pool.

Performance is unchanged for a single sequence, which is all that is
exposed for now.
2026-04-25 09:53:53 -07:00
Jesse Gross 30f86cb9dd mlxrunner: track sampler history in a fixed-size ring buffer
AppendToken used to concatenate the new token onto the history tensor
and slice it back to RepeatLastN every decode step, churning the graph
shape and reallocating a fresh tensor each call. The stateful penalties
don't care about order within the window, so a fixed-capacity ring with
one SliceUpdate per append keeps the tensor shape constant across
steps.
2026-04-25 09:53:53 -07:00
Parth Sareen ea01af6f76 openai: map responses reasoning effort to think (#15789) 2026-04-24 02:49:36 -07:00
Parth Sareen c2ebb4d57c api: accept "max" as a think value (#15787) 2026-04-24 01:49:39 -07:00
Parth Sareen 590109c835 launch: harden OpenClaw onboarding flow (#15777) 2026-04-23 16:47:20 -07:00
Eva H b4442c6d17 launch: resave managed integration config when live config drifts (#15776) 2026-04-23 19:32:36 -04:00
Eva H 85ff8e4a21 launch: keep launch recommended models in a fixed canonical order (#15750) 2026-04-23 16:33:00 -04:00
Parth Sareen 160660e572 launch: use bundled OpenClaw ollama web search (#15757) 2026-04-22 16:34:19 -07:00
madflowandParth Sareen 3b43b9bc4b docs: update structured outputs doc for cloud (#15733)
---------

Co-authored-by: Parth Sareen <parth.sareen@ollama.com>
2026-04-22 00:42:39 -07:00
Parth Sareen 21883571b7 launch: replace kimi-k2.5 with k2.6 as top recommended model (#15737) 2026-04-21 15:13:20 -07:00
Jesse Gross ce99f24731 mlxrunner: tokenize prompts in request handler goroutines
Move tokenization out of the single GPU processing goroutine and
into each request's HTTP handler goroutine. This allows the next
request's prompt to be tokenized on the CPU while the current
request is executing on the GPU.
2026-04-21 14:38:49 -07:00
Jesse Gross 04f5f0cdb4 mlx: improve thread safety of array management
Use atomic.Int32 for Array.pinned and a sync.Mutex for the global
arrays slice so MLX arrays can be created and pinned from multiple
goroutines without racing on those structures. Convert Array value
receivers to pointer receivers and struct fields from Array to
*Array to avoid copying the atomic.

This does not fully achieve thread safety even when building
completely independent graphs. The tracing flag and traceScratch
slice in compile.go are unprotected, so concurrent Compile calls
will race. MLX itself is not fully thread-safe either although
it is working to improve.
2026-04-21 14:38:49 -07:00
Matteo Celani fb36a01ffe app/ui: fix model picker showing stale model after switching chats (#15280)
* app/ui: fix model picker showing stale model after switching chats

Optimistic messages created during streaming were storing the full
Model object instead of the model name string. When switching back
to a chat with cached streaming data, the restore effect read an
object where it expected a string, causing the model picker to fail
matching and remain stuck on the previous chat's model.

* app/ui: fix two more instances of Model object passed as model name

Fix the same bug at lines 523 and 536 in the assistant_with_tools
event handler, where selectedModel (object) was used instead of
selectedModel.model (string).
2026-04-21 15:08:06 -04:00
Michael Verrilli 0c65ed33bc cmd: populate model capabilities in launchInteractiveModel (#15712)
launchInteractiveModel was introduced in PR #14609 without the
client.Show() capability-detection block that RunHandler uses.
This left opts.MultiModal always false in the TUI path, causing
image/audio file paths to always be treated as unknown commands
instead of being loaded as multimodal attachments.

Mirror the Show() call, pull-on-404 fallback, cloud auth handling,
and MultiModal/Think population from RunHandler into
launchInteractiveModel.

Fixes #15711
2026-04-21 14:37:36 -04:00
Jesse Gross 22d6c817f8 mlxrunner: fuse top-P and top-K into a single sort pass
When both filters are active, avoid paying for a full sort in top-P
and a partial sort in top-K. Single-filter paths are unchanged.
Improves generation throughput on gemma4:e4b by 1.5%.
2026-04-20 17:43:00 -07:00
Jesse Gross ca01373b28 mlxrunner: use MaxAxis in the min-P sampler
One reduction op instead of Argmax + TakeAlongAxis.
2026-04-20 17:43:00 -07:00
Jesse Gross 24e038d56a mlxrunner: add logprobs support
Match the ollamarunner and OpenAI semantics: raw, full-vocab log-softmax
with the top-K ranked by probability. Skipped on the GPU when the request
doesn't ask for logprobs so decode doesn't pay for it otherwise.
2026-04-20 17:43:00 -07:00
Parth Sareen 5d1021603a server: apply format when think=false for gemma4 (#15678) 2026-04-20 17:42:29 -07:00
Parth Sareen 8e05d734b9 launch: add kimi cli integration with installer flow (#15723) 2026-04-20 15:33:32 -07:00
Jesse Gross 05e0f21bec mlx: fuse sigmoid router head in glm4_moe_lite
DeepSeek-V2-style aux-loss-free routing computes sigmoid(gates) once but
needs it twice: the raw sigmoid output is gathered after top-k, while the
post-bias negation is the argpartition key. Fuse into a single multi-output
Compiled kernel returning both, saving two launches on the routing path
per token. Exposed as a general SigmoidRouter since the same pattern is
shared across DeepSeek-V2 descendants.

Improves glm4.7 generation performance by approximately 1%.
2026-04-20 15:02:14 -07:00
Daniel Hiltgen ff23dd343f mlx: apply repeat penalties in sampler (#15631) 2026-04-18 07:49:38 -07:00
Parth Sareen 123b300af6 docs: update hermes (#15655) 2026-04-17 14:20:59 -07:00
Parth Sareen 57653b8e42 cmd/launch: show WSL guidance on Windows instead of handing off (#15637) 2026-04-16 17:18:04 -07:00
Parth Sareen a50ce61c54 launch: skip unchanged managed-single rewrite (#15633) 2026-04-16 16:20:42 -07:00
Daniel Hiltgen 2bb7ea00d2 create: avoid gc race with create (#15628)
If you have a long running create, and start another ollama server with the
same model dir, the GC algorithm deletes the pending blobs and breaks the
create.  This adds a 1h grace period to avoid deleting in-flight creation
operations.
2026-04-16 13:29:16 -07:00
Daniel Hiltgen 55fa80d07a mlx: additional gemma4 cache fixes (#15607)
Harden additional corner cases
2026-04-16 13:07:19 -07:00
Daniel Hiltgen b9cb535407 mlx: fix gemma4 cache to use logical view (#15617) 2026-04-16 11:54:30 -07:00
Daniel Hiltgen 031baef094 mlx: fix imagegen lookup (#15588)
* mlx: fix imagegen lookup

Fixes #15533 - imagegen had fallen out of sync with the new layout
for multiple mlx libraries on Metal.

* review comments
2026-04-16 10:39:00 -07:00
7d271e6dc9 cmd/launch: add Copilot CLI integration (#15583)
---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-04-15 17:22:53 -07:00
Devon Rifkin c88dae2d6b Merge pull request #15612 from ollama/drifkin/gemma4-split-templates
gemma4: render differently based on model size
2026-04-15 17:15:35 -07:00
Devon Rifkin 9e3618d663 make empty block conditional 2026-04-15 15:35:25 -07:00
Daniel Hiltgen 5d920cc6bc Keep Gemma4 router projection in source precision (#15613) 2026-04-15 15:04:23 -07:00
Devon Rifkin e585ecd11f gemma4: render differently based on model size
Following up on #15560, this change now has e2b/e4b render differently
from 26b/31b.

For backwards compatibility, we take the existing renderer name `gemma4`
and make it do dynamic resolution based on the model name/size, but the
intended use is for the models to be republished with the renderer
variant specified explicitly: `gemma4-small` or `gemma4-large`.
2026-04-15 14:37:16 -07:00
Eva H cdddea0592 launch: always list cloud recommendations first (#15593) 2026-04-15 13:17:35 -07:00
Parth Sareen 43f90def04 launch: add hermes (#15569) 2026-04-15 12:00:23 -07:00
Daniel Hiltgen 06ae6367bd mlx: fix RotatingKVCache.concat() dropping context on mid-rotation (#15591)
After the rotating buffer has wrapped (c.offset > c.maxSize) a subsequent
L>1 Update() went through a slice-to-[0, c.idx) path that discarded all
slots in [c.idx, Dim), losing the older-but-still-in-window tokens the
first Q of the new batch needs for its sliding-window attention.

Linearize the circular buffer to logical order in that wrapped case so
the existing trim + concat preserves the last (maxSize - 1) old tokens.
When the buffer has not yet wrapped (c.offset <= c.maxSize), slots
[c.idx, Dim) are grow padding or stale post-rewind data, so keep
dropping them.
2026-04-14 18:29:06 -07:00
Daniel Hiltgen 48ad7085c4 mlx: Improve gemma4 performance with fused operations (#15587)
* mlx: Improve gemma4 performance with fused operations

* review comments
2026-04-14 18:04:04 -07:00
Jesse Gross e1e3cec8d0 models: fuse MLP activation functions via mlx_compile
Converts SiLU/GELUApprox to compiled kernels and adds SwiGLU,
matching upstream mlx/mlx_lm's activations pattern. Routes llama,
qwen3, qwen3_5 (dense + MoE), and glm4_moe_lite MLP paths through
mlx.SwiGLU so each MLP invocation runs as one fused Metal/CUDA
kernel rather than a chain of per-op launches.
2026-04-14 16:38:32 -07:00
Jesse Gross d3e67e305c mlx: add compiled closure support
Wraps MLX's mlx_compile API so Go functions can be traced into fused
kernels. Contiguous elementwise chains collapse into a single
Metal/CUDA kernel instead of launching one per op.

Exposes Compile plus arity helpers (Compile1/2/3) that mirror Python's
@mx.compile decorator shape, lazily building the closure on first call
so package-level declarations work before the MLX dylib loads.
2026-04-14 16:38:32 -07:00
Eva H 698e04a14b launch: OpenCode inline config (#15586) 2026-04-14 15:08:42 -07:00
Eva H 1d9537bc33 launch/openclaw: fix --yes flag behaviour to skip channels configuration (#15589) 2026-04-14 13:57:35 -07:00
Eva H 120424d832 Revert "launch/opencode: use inline config (#15462)" (#15568) 2026-04-13 18:40:17 -07:00
Eva H 5818001610 launch: skip unchanged integration rewrite configration (#15491) 2026-04-13 17:18:56 -07:00
Daniel Hiltgen 2cba7756c5 Gemma4 on MLX (#15244)
* gemma4: implement Gemma 4 model for MLX (text-only runtime)

* gemma4: two MoE + SWA prefill perf fixes

Two performance optimizations in the gemma4 forward pass

1. Memoize the sliding-window prefill mask across layers.
2. Softmax only over the selected experts in Router.Forward.

* review comments
2026-04-13 16:36:51 -07:00
Devon Rifkin bf2a421727 gemma4: restore e2b-style nothink prompt (#15560)
Gemma 4 prompts differ when thinking is disabled for different sized
models: 26b/31b emit an empty thought block, while e2b/e4b do not.

Before #15490, our shared Gemma 4 renderer effectively matched the
e2b behavior. #15490 changed it to always emit the empty thought block,
which regressed e2b/e4b nothink behavior and led to #15536 (and possibly

This change restores the previous shared behavior by removing the empty
trailing thought block. It also renames the checked-in upstream chat
templates so the e2b and 31b fixtures are tracked separately.

A follow-up will split Gemma 4 rendering by model size.

Fixes: #15536
2026-04-13 14:26:15 -07:00
Eva H f3cf6b75fb launch/opencode: use inline config (#15462) 2026-04-13 13:41:31 -07:00
Devon Rifkin 5dfac387a6 Revert "gemma4: fix nothink case renderer (#15553)" (#15556)
This reverts commit 4d75f5da03.
2026-04-13 13:12:18 -07:00
Daniel Hiltgen a99e5d9c22 mac: prevent generate on cross-compiles (#15120)
For some versions of Xcode, cmake builds are failing due to header problems in
cross-compiling during the generate phase.  Since generate is producing arch
independent generated output, we can skip this during cross-compiling.
2026-04-13 13:04:58 -07:00
Daniel Hiltgen 0abf3aca36 cgo: suppress deprecated warning to quiet down go build (#15438) 2026-04-13 13:04:11 -07:00
Devon Rifkin ee0266462a Revert "gemma4: add nothink renderer tests (#15554)" (#15555)
This reverts commit 1b70bb8a10.
2026-04-13 13:00:59 -07:00
Daniel Hiltgen c88fb286ec mlx: add op wrappers for Conv2d, Pad, activations, trig, and masked SDPA (#14913)
* mlx: add op wrappers for Conv2d, Pad, activations, trig, and masked SDPA

Add Conv2d, flexible Pad (with axes/mode), PadConstant, Maximum,
Minimum, Softplus, ReLU, GLU, Clamp, Sin, Cos, Clip,
ScaledDotProductAttentionMasked, and RoPEWithFreqs. Refactor
RoPEWithBase to delegate to RoPEWithFreqs.

* review comments

* mlx: fix ScaledDotProductAttentionMasked to consult the mask argument
2026-04-13 11:43:24 -07:00
Daniel Hiltgen d3da29cbfc mlx: mixed-precision quant and capability detection improvements (#15409)
Improve the MLX model creation pipeline with several model-agnostic changes:

- Rewrite supportsVision to use vision_config instead of architecture name
- Add supportsAudio for audio encoder detection
- Add alignment checking (isAligned) for quantization group sizes
- Support per-projection mixed quantization in MoE expert packing
- Record per-tensor quant metadata in safetensors blobs
- Parse per-tensor quant metadata at model load time
- Validate quantize output is non-empty before storing
- Fix pin/unpin cleanup in expert group quantization
- Promote v_proj/k_proj/down_proj to INT8 for INT4 base quant
- Add MetalIsAvailable() utility
- Skip audio encoder tensors from quantization
2026-04-13 11:43:07 -07:00
Devon Rifkin 1b70bb8a10 gemma4: add nothink renderer tests (#15554)
Meant to include in #15553
2026-04-13 11:38:19 -07:00
Daniel Hiltgen ec29ce4ce3 gemma4: fix compiler error on metal (#15550)
On some systems, the metal runtime compiler is failing due to an
uninitialized variable from #15378.

Fixes #15548
2026-04-13 11:32:00 -07:00
Devon Rifkin 4d75f5da03 gemma4: fix nothink case renderer (#15553)
Regressed in #15490

Fixes: #15536
2026-04-13 11:23:19 -07:00
saman-amdandSamiii777 798fd09bfe Update to ROCm 7.2.1 (#15483)
Co-authored-by: Samiii777 <58442200+Samiii777@users.noreply.github.com>
2026-04-12 12:11:58 -07:00
Devon Rifkin 9330bb9120 gemma4: be less strict about whitespace before bare keys (#15494) 2026-04-11 16:30:27 -07:00
Devon Rifkin 40a1317dfd gemma4: update renderer to match new jinja template (#15490)
* gemma4: update renderer to match new jinja template

Google has updated their jinja template for gemma4, and so this change
gives us parity with the new template. The parsing also slightly changed
upstream, so we make a small change to our parser as well.

I've also corrected a few probably existing edge cases, especially
around type unions. The upstream output format is weird (a stringified
array), but in practice the models seem to understand it well.

* gemma4: special case simple `AnyOf`s

The upstream template doesn't handle `AnyOf`s, but since in the previous
commit we saw type unions work reasonably well, I'm now treating very
simple `AnyOf`s as type unions to help in cases where they might be used

* fix lint

* gemma4: prefer empty instead of `None`

We can't currently distinguish between a result being not-present vs.
empty. The empty case seems more important (e.g., a legitimately empty
tool call)

* gemma4: be more careful for tool results with missing IDs
2026-04-10 15:45:27 -07:00
Devon Rifkin fdfe9cec98 model/parsers: fix missing parallel tool call indices (#15467)
We were missing setting the function index for several models that can
make parallel tool calls.

In the future we may want to consider putting some sort of post-parse
hook and relieve the parsers of this duty.

Fixes: #15457
2026-04-10 15:23:21 -07:00
Matteo Celani 9517864603 app/ui: re-validate image attachments when selected model changes (#15272) 2026-04-10 14:03:51 -07:00
Bruce MacDonald 8e6d86dbe3 docs: add hermes agent integration guide (#15488)
Update cloud and local model recommendations to match current
models.go: add qwen3.5:cloud and glm-5.1:cloud, replace glm-4.7-flash
with gemma4 and qwen3.5 as local options.

Add documentation for Hermes Agent by Nous Research, covering
installation, Ollama setup via custom endpoint, messaging configuration,
and recommended models.
2026-04-10 13:13:36 -07:00
Parth Sareen 80d3744c5d launch: update openclaw channel message (#15463) 2026-04-09 15:20:30 -07:00
Eva H 2a94f03823 launch: add re-run hint to dependency error message (#15439) 2026-04-09 09:51:34 -07:00
Patrick Devine eb97274e5c modelfiles: fix /save command and add shortname for safetensors based models (#15413)
This change fixes two issues with Modelfiles:

  1. If a user uses `ollama show --modelfile` to show a safetensors based
     model, the Model would leave the "FROM" field blank which won't allow
     a user to recreate the model. This change adds the model's current
     canonical short name to the FROM field.
  2. If a user uses the `/save` command in the CLI any messages which were
     saved in a previous model wouldn't get saved (only the set of messages
     from the current session).
2026-04-08 21:05:39 -07:00
Daniel Hiltgen 6b5db12aa2 mlx: remove stale x86 libmlx library (#15443)
Fixes #15433
2026-04-08 20:51:47 -07:00
7. SunandClaude Sonnet 4.6 612f0a17d3 fix: improve error message for unknown input item type in responses API (#15424)
The default branch in unmarshalResponsesInputItem had two issues:
- It referenced typeField.Type instead of itemType; these differ when the
  shorthand role-based format promotes an empty type to "message", meaning
  an unhandled type would show the wrong value in the error string.
- It used %s formatting, so an empty type field produced the unhelpful
  message "unknown input item type: " with no indication what was missing.

Fix by using itemType (the resolved value) with %q quoting, and add a
dedicated message when itemType is empty (both type and role absent):
"input item missing required 'type' field".

Tests added for the empty-type and missing-type cases.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 17:41:12 -07:00
Parth Sareen 673726fa0e app: restore launch default and refine launch sidebar open for app (#15437) 2026-04-08 16:59:21 -07:00
Daniel Hiltgen b5918f9785 pull/push: refine safetensors (#14946)
* pull: refine safetensors pull

 - Body drain in resolve() — drain response body before close so Go's HTTP
   client can reuse TCP connections instead of opening a new one per blob
   (1,075 extra TCP+TLS handshakes eliminated)
 - Skip speed recording for tiny blobs (<100KB) — prevents
   HTTP-overhead-dominated transfer times from poisoning the median, which the
   stall detector uses to cancel "too slow" downloads
 - Resume support for large blobs (>=64MB) — on failure, preserves partial .tmp
   files; on retry, re-hashes existing datak and sends Range header to download
   only remaining bytes; gracefully falls back to full download if server returns
   200 instead of 206; SHA256 verification catches corrupt partials

* harden push

- Prevents killing TCP connections after every request.
- Stronger backoff to handle server back-pressure and rate limiting
- Larger buffered reads for improve safetensor upload performance
- Better error message handling from server
- Handle 201 if server says blob exists
- Fix progress reporting on already uploaded blobs
- Trace logging to help troubleshoot and tune going forward

* review comments

* review comments
2026-04-08 14:15:39 -07:00
Eva H d17f482d50 launch/opencode: detect curl installed opencode at ~/.opencode/bin (#15197) 2026-04-08 13:54:51 -07:00
Parth Sareen 4e16f562c0 launch: add openclaw channels setup (#15407) 2026-04-08 13:25:27 -07:00
Parth Sareen 55308f1421 launch: update ctx length for glm-5.1 and gemma4 (#15411)
Also adds glm-5.1 in recommended models
2026-04-08 12:11:50 -07:00
Eva H d64812eb5d cmd: improve multi-select sorting and selection status (#15200) 2026-04-08 10:39:18 -07:00
Devon Rifkin f86a969f27 responses: add support for fn call output arrays (#15406)
In addition to strings (which we already supported), OpenResponses
supports arrays of text content, image content, or file content (see
<https://www.openresponses.org/reference#object-FunctionCallOutput-title>).
We were missing support for these arrays, which caused unmarshal errors
like

```
json: cannot unmarshal array into Go struct field ResponsesFunctionCallOutput.output of type string
```

This change adds support for text content and image content, as those
are more straightforwardly mappable to Ollama message formats (though
image and text interleaving is lost), but it's less clear what to do for
files. In the future we can partially support this by inlining
reasonably sized text files, but wanted to get this change out first.

Fixes: #15250
2026-04-07 16:47:30 -07:00
Matteo Celani 9fa80a1660 app/ui: fix lint errors for unused vars, prefer-const, and empty catch (#15282) 2026-04-07 16:28:36 -07:00
Daniel Hiltgen dde09129d1 gemma4: Disable FA on older GPUs where it doesn't work (#15403)
CUDA older than 7.5 lack the support to enable flash attention for the model.
2026-04-07 14:54:25 -07:00
Patrick Devine 780556c4d0 mlx: use default http client (#15405) 2026-04-07 14:53:23 -07:00
Daniel Hiltgen dfae363b5b gemma4: add missing file (#15394)
File accidentally omitted from #15378
2026-04-07 09:18:01 -07:00
Daniel Hiltgen 30fdd229a4 create: Clean up experimental paths, fix create from existing safetensor model (#14679)
* create:  Clean up experimental paths

This cleans up the experimental features, and adds both unit and integration test coverage to verify no regressions.

* create: preserve config and layer names when creating from safetensors models

When creating a model FROM an existing safetensors model, ModelFormat,
Capabilities, and layer Name fields were lost. ModelFormat stayed empty
because it's only set from GGML layers (which safetensors models lack),
and layer names weren't copied in parseFromModel. This caused derived
models to fail loading ("config.json not found in manifest").

* review comments
2026-04-07 08:12:57 -07:00
Daniel Hiltgen e823bff873 gemma4: enable flash attention (#15378)
Backport GGML kernels so we can enable flash attention for the gemma 4 model on
Metal and CUDA.
2026-04-07 08:12:36 -07:00
Daniel Hiltgen 8968740836 mlx: Improve M5 performance with NAX (#15345)
* mlx: Improve M5 performance with NAX

This modifies the Mac release to now have 2 builds of MLX for broader
compatibility while supporting the latest M5 hardware features.  NAX requires
building with xcode 26.2 and targetting support only for OS v26 and up.  Since
we want to support older MacOS versions as well, we now need 2 different MLX
builds and runtime detection logic to select the optimal version.  The newer
build will detect NAX missing at runtime, so it is safe to run on pre M5 macs.

* mac: prevent generate on cross-compiles

For some versions of Xcode, cmake builds are failing due to header problems in
cross-compiling during the generate phase.  Since generate is producing arch
independent generated output, we can skip this during cross-compiling.
2026-04-07 08:12:24 -07:00
Devon Rifkin 8c8f8f3450 model/parsers: add gemma4 tool call repair (#15374)
The existing strict gemma4 tool parser is still the primary path, but if
this fails, we try to repair by fixing some of the most commonly seen
mistakes these models seem to make in practice.

We repair by building up a set of candidates, and use the first candidate
that parses.

Repairs cover:

- missing Gemma string delimiters
- single-quoted string values, including a dangling Gemma delimiter
- raw terminal string values (if the corresponding tool schema indicates
  it should be a string)
- missing object close only after a concrete repair

Add regression coverage for malformed tool calls from issue #15315 and
focused unit tests for the individual repair helpers and candidate
pipeline.
2026-04-06 18:47:17 -07:00
Parth Sareen 82f0139587 launch/openclaw: patch approvedScopes baseline for TUI pairing (#15375) 2026-04-06 18:00:12 -07:00
Bruce MacDonald 26a58b294c app: update featured models (#15373)
Featured models in the app are out of date. Update them to a more recent list of models.
2026-04-06 16:35:35 -07:00
Devon Rifkin 34a790a2e6 model/parsers: suppress extra gemma4 closing tool tags (#15370)
We've observed Gemma 4 occasionally emitting extra <tool_call|> tags
after a valid tool call. We suppress leading close tags in this
immediate post-tool-call state so the extra close tags do not leak into
assistant content. The tradeoff is that if the model intentionally
begins its next content span with the literal string "<tool_call|>", we
will erroneously treat it as noise and drop it.
2026-04-06 12:41:33 -07:00
Jeffrey Morgan 4589fa2cf5 app: default app home view to new chat instead of launch (#15312) 2026-04-03 21:50:55 -07:00
Daniel Hiltgen 4bc2728047 Revert "enable flash attention for gemma4 (#15296)" (#15311)
This reverts commit c8e0878814.
2026-04-03 17:44:44 -07:00
Devon Rifkin 49d5fd5a3e model/parsers: rework gemma4 tool call handling (#15306)
Replace the custom Gemma4 argument normalizer with a stricter
reference-style conversion: preserve Gemma-quoted strings, quote bare
keys, and then unmarshal the result as JSON.

This keeps quoted scalars as strings, preserves typed unquoted values,
and adds test coverage for malformed raw-quoted inputs that the
reference implementation rejects.
2026-04-03 14:35:00 -07:00
Jesse Gross 3cd2b03a5e ggml: fix ROCm build for cublasGemmBatchedEx reserve wrapper
Add missing cublasGemmAlgo_t to hipblasGemmAlgo_t type mapping and
cast away const qualifiers that hipblasGemmBatchedEx doesn't accept.
2026-04-03 14:22:46 -07:00
Daniel Hiltgen c8e0878814 enable flash attention for gemma4 (#15296) 2026-04-03 12:46:18 -07:00
Jesse Gross bb0c58e134 ggml: skip cublasGemmBatchedEx during graph reservation
cublasGemmBatchedEx fails during graph capture when pool allocations
return fake pointers. This is triggered when NUM_PARALLEL is greater
than 1 for models like gemma4 that use batched matmuls. Skip it
during reservation since the memory tracking is already handled by
the pool allocations.

Fixes #15249
2026-04-03 12:41:09 -07:00
Devon RifkinandCharles H 036ed1b9b5 model/parsers: fix gemma4 arg parsing when quoted strings contain " (#15254)
* model/parsers: fix gemma4 arg parsing when quoted strings contain "

Fixes: #15241

* add more tests, be careful about what we escape

We want Windows-style paths to not get misinterpreted

* fix backslash-quote case, it really should be a literal backslash

h/t to @chathaway-codes for pointing this out!

Co-Authored-By: Charles H <2773397+chathaway-codes@users.noreply.github.com>

---------

Co-authored-by: Charles H <2773397+chathaway-codes@users.noreply.github.com>
2026-04-02 22:52:51 -07:00
Daniel Hiltgen 3536ef58f6 bench: add prompt calibration, context size flag, and NumCtx reporting (#15158)
Add --num-ctx flag to set context size, and report NumCtx in model info
header. Calibrate tokens-per-word ratio during warmup using actual
tokenization metrics from the model, replacing the fixed 1.3 heuristic.
This produces more accurate prompt token counts for --prompt-tokens.

Also add fetchContextLength() to query running model context via /api/ps.
2026-04-02 14:23:53 -07:00
Daniel Hiltgen de9673ac3f tokenizer: add byte fallback for SentencePiece BPE encoding (#15232)
* tokenizer: add byte fallback for SentencePiece BPE encoding

When BPE merging produces tokens not in the vocabulary, fall back to
encoding each UTF-8 byte as <0xHH> byte tokens instead of silently
dropping the character. Also teach Decode to convert <0xHH> tokens
back to raw bytes.

Fixes #15229, fixes #15231

* tokenizer fixes
2026-04-02 13:04:45 -07:00
Daniel Hiltgenandjmorganca 96b202d34b Add support for gemma4 (#15214)
* bench: add prompt calibration, context size flag, and NumCtx reporting

Add --num-ctx flag to set context size, and report NumCtx in model info
header. Calibrate tokens-per-word ratio during warmup using actual
tokenization metrics from the model, replacing the fixed 1.3 heuristic.
This produces more accurate prompt token counts for --prompt-tokens.

Also add fetchContextLength() to query running model context via /api/ps.

* integration: improve vision test robustness and add thinking tests

Add skipIfNoVisionOverride() to skip vision tests when OLLAMA_TEST_MODEL
is set to a non-vision model. Add Think:false to context exhaustion test
to prevent thinking models from using all context before the test can
measure it. Add third test image (ollama homepage) and replace OCR test
with ImageDescription test using it. Relax match strings for broader
model compatibility. Add TestThinkingEnabled and TestThinkingSuppressed
to verify thinking output and channel tag handling.

* gemma4: add Gemma 4 GGML model support

Add full Gemma 4 model family support (E2B, E4B, 26B MoE, 31B Dense)
for the GGML backend including text, vision, converter, parser, and
renderer.

Text model features:
- Sliding window + full attention with per-layer patterns
- KV sharing across layers with donor map
- Per-layer embeddings (PLE) with learned projections
- MoE routing with RMSNorm + learned scale
- Proportional RoPE with freq_factors for global attention
- Final logit softcapping

Vision model features:
- SigLIP vision encoder with 2D RoPE
- ClippableLinear with input/output clamping via packed v.clamp_data
- Adaptive average pooling with nMerge kernel
- Multi-modal projection with unweighted RMSNorm

Converter:
- Safetensors to GGUF with vision tensor renaming
- Fused MoE gate_up_proj splitting
- Vision patch embedding reshape (HF to Conv2D layout)
- Packed clamp data tensor for ClippableLinear bounds
- Proportional RoPE freq_factors generation

Also includes:
- BackendGet() on ml.Tensor for reading weight tensor data
- Q6_K CUDA get_rows kernel support
- MoE-aware ffn_down quantization layer counting
- Gemma4 parser with tool calling and thinking support
- Gemma4 renderer with structured tool format
- Architecture-based auto-detection of renderer/parser/stop tokens
- Integration test gemma4 model list additions

* gemma4: add audio support with USM conformer encoder

Add audio encoding for Gemma 4 using the USM conformer architecture:
- Converter: audio tensor mapping, SSCP/conformer/embedder name replacements,
  softplus repacker for per_dim_scale, F32 enforcement for conv weights
- GGML backend: Conv1DDW and PadExt tensor ops
- Audio encoder: SSCP Conv2D, 12 conformer blocks (FFW + block-local
  attention with relative position embeddings + LightConv1d + FFW),
  output projection, audio-to-text embedding projector
- Audio preprocessing: WAV decode, mel spectrogram, FFT (pure Go)
- Model wiring: WAV detection, audio token handling, unified PostTokenize

Correctly transcribes "why is the sky blue" from test audio.

* integration: add gemma4 audio tests including OpenAI API coverage

Test audio transcription and response via the Ollama native API, plus
two new tests exercising the OpenAI-compatible endpoints:
- /v1/audio/transcriptions (multipart form upload)
- /v1/chat/completions with input_audio content type

All tests use capability checks and skip models without audio support.

* gemma4: add OpenAI audio API support and capability detection

- Add CapabilityAudio and detect from audio.block_count in GGUF
- Add /v1/audio/transcriptions endpoint with TranscriptionMiddleware
- Add input_audio content type support in /v1/chat/completions
- Add TranscriptionRequest/Response types in openai package

* gemma4: add audio input support for run command

- /audio toggle in interactive mode for voice chat
- Platform-specific microphone recording (AVFoundation on macOS,
  PulseAudio/ALSA on Linux, WASAPI on Windows)
- Space to start/stop recording, automatic chunking for long audio

* gemma4: add transcribe command (ollama transcribe MODEL)

- Interactive mode with readline prompt and slash commands
- Non-interactive mode for piped audio or record-until-Ctrl+C
- Chunked streaming transcription for long recordings
- Word-wrapped output matching run command style

* gemma4: add parser, renderer, and integration test plumbing

* gemma4: fix renderer to emit BOS token

* gemma4: add OpenAI audio transcription API and input_audio support

* gemma4: update converter for new weight drop naming

* gemma4: add per_expert_scale to MoE router and fix moe_intermediate_size config

* gemma4: rewrite renderer to match HF Jinja2 template exactly

Fix 8 bugs found by building 55 reference tests verified against the
HF Jinja2 chat template (VERIFY_JINJA2=1 shells out to Python):

- Tool responses use separate <|turn>tool turns (not inline tags)
- Tool calls emitted before content in assistant messages
- Thinking content stripped from assistant history (strip_thinking)
- User, tool, and system content trimmed (template does | trim)
- Empty system message still emits system turn (check role, not content)
- Nested object properties rendered recursively with required field
- Array items specification rendered for array-type properties
- OBJECT/ARRAY type-specific rendering comma logic matches template

Also adds Required field to api.ToolProperty for nested object schemas,
replaces old gemma4_test.go with comprehensive gemma4_reference_test.go,
and commits the Jinja2 template as testdata for verification.

* gemma4: fix MoE fused gate_up split and multiline tool-call arg parsing

- Text MoE: split `ffn_gate_up_exps` into contiguous `[gate|up]` halves instead of stride-2 slices.
- Parser: escape control characters in `<|"|>...<|"|>` string literals when converting tool-call args to JSON.
- Fixes warnings like `invalid character '\n' in string literal` for multiline tool arguments.
- Add Gemma4 parser regressions for multiline tool-call args and `gemma4ArgsToJSON`.

* cmd: simplify audio input to dropped file attachments

* gemma4: use full SWA memory for better cache reuse

* gemma4: initialize clamps after backend load

* convert: align gemma4 audio tensor renames with llama.cpp

* Remove redundant comments in gemma4 vision model

* Format Gemma4 MoE block field alignment

* use 4096 kvcache.NewSWAMemCache

* convert: support new Gemma4 audio_tower tensor naming (#15221)

Co-authored-by: jmorganca <jmorganca@gmail.com>

* fix integration test defaults for audio

* review comments and lint fixes

* remove unused audio/video files

---------

Co-authored-by: jmorganca <jmorganca@gmail.com>
2026-04-02 11:33:33 -07:00
Devon Rifkin 79865e6c5a app: use the same client for inference and other requests (#15204)
Previously we were accidentally using different clients/UAs depending on
whether it was an inference call or a different call. This change makes
them consistent, other than the timeout being different.
2026-04-02 11:07:50 -07:00
Parth Sareen 5ab10d347a app: add launch page for a simple way to launch integrations (#15182) 2026-04-02 10:31:19 -07:00
Eva H a8292dd85f launch: replace deprecated OPENAI_BASE_URL with config.toml profile for codex (#15041) 2026-04-01 11:43:23 -04:00
Daniel Hiltgen cb0033598e tokenizer: add SentencePiece-style BPE support (#15162)
* tokenizer: add SentencePiece-style BPE support

Add WithSentencePieceNormalizer option to BytePairEncoding for models
that use BPE with SentencePiece-style space markers (space to/from
U+2581).

NewBytePairEncoding is unchanged; the new NewBytePairEncodingWithOptions
constructor accepts BPEOption functions. Decoding handles the reverse
mapping of U+2581 back to spaces.

* review comments
2026-03-31 17:00:36 -07:00
Daniel Hiltgen 4d14b0ff92 mlx: respect tokenizer add_bos_token setting in pipeline (#15185)
Replace hardcoded Encode(prompt, true) with
Encode(prompt, r.Tokenizer.AddBOS()) so the pipeline respects each
model's tokenizer configuration.

Models with add_bos_token=true (gemma3, llama): unchanged, tokenizer
still prepends BOS.

Models with bos_token=null (qwen3, qwen3.5): unchanged, the BOS
guard (vocab.BOS >= 0) already prevented prepending regardless of
the flag.

This aligns the pipeline with the /v1/tokenize endpoint which already
uses Tokenizer.AddBOS().
2026-03-31 16:46:30 -07:00
Parth Sareen d9cb70c270 docs: update pi docs (#15152) 2026-03-31 16:37:55 -07:00
Jeffrey Morgan 31f968fe1f cmd: set OpenCode default model in config (#15127) 2026-03-29 12:11:36 -07:00
Jeffrey Morgan b7bda92d52 model: add qwen3-next compatibility for legacy ssm_in projections (#15133) 2026-03-29 11:50:47 -07:00
Parth Sareen 8e54823fd3 revert context length warnings change (#15121) 2026-03-28 16:43:59 -07:00
Parth Sareen 7c8da5679e launch: improve multi-select for already added models (#15113) 2026-03-28 13:44:40 -07:00
Parth Sareen 6214103e66 launch: auto-install pi and manage web-search lifecycle (#15118) 2026-03-28 13:06:20 -07:00
Patrick Devine 9e7cb9697e mlx: fix vision capability + min version (#15106) 2026-03-27 17:09:28 -07:00
Bruce MacDonald 3824e380a8 server: preserve raw manifest bytes during pull (#15104)
pullModelManifest unmarshals the registry response into a Go struct
then re-marshals with json.Marshal before writing to disk. When the
registry's JSON formatting or field ordering differs from Go's
output, the local SHA256 won't match the registry's
Ollama-Content-Digest header, causing false "out of date" warnings.

Preserve the raw bytes from the registry response and write them
directly to disk so the local manifest is byte-for-byte identical
to what the registry serves.
2026-03-27 15:42:31 -07:00
Devon Rifkin c9b2dcfc52 anthropic: fix empty inputs in content blocks (#15105)
* anthropic: fix empty inputs in content blocks

When we switched to `api.ToolCallFunctionArguments`, `omitempty` stopped
doing what we were relying on it for before. This would cause non-tool
content blocks to have an `"input": {}` field, which doesn't match our
old behavior.

* use omitzero instead
2026-03-27 15:41:27 -07:00
Parth Sareen b00bd1dfd4 launch: skip context length warning for MLX models and show model name (#15102) 2026-03-27 15:01:33 -07:00
Jesse Gross ac83ac20c4 anthropic: fix KV cache reuse degraded by tool call argument reordering
Use typed structs for tool call arguments instead of map[string]any to
preserve JSON key order, which Go maps do not guarantee.
2026-03-27 14:30:16 -07:00
Bruce MacDonald e7ccc129ea app: fix false "out of date" model warnings (#15101)
The staleness check compared the local manifest digest (SHA256 of the
file on disk) against the registry's Ollama-Content-Digest header.
These never matched because PullModel re-serializes the manifest JSON
before writing, producing different bytes than the registry's original.

The fallback comparison (local modified_at vs upstream push time) was
also broken: the generated TypeScript Time class discards the actual
timestamp value, so Date parsing always produced NaN.

Fix by moving the staleness comparison server-side where we have
reliable access to both the local manifest file mtime and the upstream
push time. The /api/v1/model/upstream endpoint now returns a simple
`stale` boolean instead of raw digests for the frontend to compare.

Also adds User-Agent to the CORS allowed headers for dev mode.
2026-03-27 14:15:10 -07:00
Jeffrey Morgan 69ed0c2729 parsers: qwen3.5 streaming tool-call parsing and add regression test (#15098) 2026-03-27 14:04:14 -07:00
Alfredo Matas 1cefa749aa model/parsers: close think block if tool block starts in Qwen3.5 (#15022) 2026-03-27 11:28:34 -07:00
Daniel Hiltgen aec2fef95d ci: harden cuda include path handling (#15093)
On windows we can get multiple include dirs, so find where the headers are then
copy from that location.
2026-03-27 07:57:07 -07:00
Eva HandParth Sareen 366625a831 launch: warn when server context length is below 64k for local models (#15044)
A stop-gap for now to guide users better. We'll add more in-depth recommendations per integration as well.

---------

Co-authored-by: Parth Sareen <parth.sareen@ollama.com>
2026-03-27 00:15:53 -07:00
Daniel Hiltgen 516ebd8548 ci: include mlx jit headers on linux (#15083)
* ci: include mlx jit headers on linux

* handle CUDA JIT headers
2026-03-26 23:10:07 -07:00
Parth Sareen f567abc63f tui: update chat title (#15082) 2026-03-26 18:06:53 -07:00
Eva H 1adfc27f04 launch/vscode: prefer known vs code paths over code on PATH (#15073) 2026-03-26 18:06:28 -04:00
Parth Sareen 4a2b9f9dbc launch: hide cline integration (#15080) 2026-03-26 14:33:43 -07:00
Parth Sareen e46b67a6cc launch: hide vs code (#15076) 2026-03-26 13:52:50 -07:00
Eva HandParthSareen c000afe76c doc: update vscode doc (#15064)
---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-03-26 13:45:48 -07:00
Jesse Gross 9d7b18f81e mlxrunner: combine setStateRaw and setStateDetached into setState 2026-03-26 13:32:11 -07:00
Jesse Gross 4f5999fd3f mlxrunner: schedule periodic snapshots during prefill
Add periodic snapshots every 8k tokens and near the end of the prompt
so that long prompts can be partially restored and thinking/generation
can be retried without full reprocessing.
2026-03-26 13:32:11 -07:00
Jesse Gross ac5f0dbb6a mlxrunner: improve eviction and LRU tracking
Update LRU last used time just on the nodes that actually used
during processing rather than all snapshots along the path. This
allows eviction to remove nodes more accurately so we can avoid
other heuristics to auto-merge nodes.
2026-03-26 13:32:11 -07:00
Jesse Gross d1151e18a1 mlx: fix KV cache snapshot memory leak
mlx.Copy shares the backing buffer with its source (via
copy_shared_buffer) rather than allocating independent storage.
When used to snapshot a slice of the KV cache, the snapshot array
holds the entire original cache buffer alive through the shared
data pointer — even after eval detaches the computation graph.

Replace Copy with Contiguous in Snapshot and Split. Contiguous
allocates a compact buffer when the source buffer is significantly
larger than the logical slice (Contiguous::eval checks
buffer_size > nbytes + 16384), which is always the case for KV
cache slices.
2026-03-25 17:26:34 -07:00
rick ebbce136c7 ggml: force flash attention off for grok 2026-03-25 16:15:49 -07:00
Devon Rifkin 26b9f53f8e api/show: overwrite basename for copilot chat (#15062)
Copilot Chat prefers to use `general.basename` in the built-in Ollama
integration, but this name isn't usually shown directly to users (and
there may be many models that share this name). Instead we pass back
`req.Model`, which for this extension is the value that we return from
`/api/tags`
2026-03-25 14:02:22 -07:00
Eva HandParth Sareen 7575438366 cmd: ollama launch vscode (#15060)
Co-authored-by: Parth Sareen <parth.sareen@ollama.com>
2026-03-25 16:37:02 -04:00
Eva H 7d7c90d702 tui: add left arrow back navigation in model selector (#14940) 2026-03-25 11:53:48 -07:00
Daniel Hiltgen 4fda69809a ci: fix windows cgo compiler error (#15046) 2026-03-24 16:45:36 -07:00
Daniel Hiltgen c9b5da6b0c integration: improve ability to test individual models (#14948)
* integration: improve ability to test individual models

Add OLLAMA_TEST_MODEL env var to run integration tests against a
single model.

Enhance vision tests: multi-turn chat with cached image tokens, object
counting, spatial reasoning, detail recognition, scene understanding, OCR, and
multi-image comparison.

Add tool calling stress tests with complex agent-style prompts, large
system messages, and multi-turn tool response handling.

* review comments
2026-03-24 14:28:23 -07:00
Patrick Devine de5cb7311f mlx: add mxfp4/mxfp8/nvfp4 importing (#15015)
This change allows importing bf16 and converting to mxfp4/mxfp8/nvfp4
and also importing fp8 and converting directly to mxfp8.
2026-03-24 13:45:44 -07:00
Jesse Gross 95ee7fbd29 mlxrunner: panic on double unpin 2026-03-23 17:44:19 -07:00
Jesse Gross ec55536734 mlxrunner: show time since last used in cache dump tree 2026-03-23 17:44:19 -07:00
Jesse Gross 77491439c2 mlxrunner: support partial match on pure transformer caches
Previously, a partial match within a node's edge would truncate the path
to the parent snapshot - effectively making all cache types behave as
recurrent caches. Caches with only transformer layers can rewind to
arbitrary boundary so this restores this capability to improve cache
hits
2026-03-23 17:44:19 -07:00
Parth Sareen b166b36cd2 docs: update Claude Code with Telegram guide (#15026) 2026-03-23 16:31:21 -07:00
Daniel Hiltgen c2b0bb7a52 mlx: update as of 3/23 (#14789)
* mlx: update to HEAD on 3/23

Also fixes a few misc vendoring bugs uncovered with this first update.
This also renames the version files to make them clearer.

* CUDA Fast Gated Delta kernel

* mlx: detect eval errors and panic

On model errors or missing kernels, don't mask the error, bubble it up.
2026-03-23 11:28:44 -07:00
Bruce MacDonaldandParthSareen 22c2bdbd8a docs: nemoclaw integration (#14962)
---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-03-20 15:27:37 -07:00
Bruce MacDonald 6df6d097d9 launch: skip openclaw gateway health check when no daemon install (#14984) 2026-03-20 15:20:14 -07:00
Jesse Gross d7c176ab91 llm, mlxrunner: fix done channel value consumed by first receiver
Receiving from a buffered chan error consumes the value, so only the
first caller (WaitUntilRunning, HasExited, or Close) sees the signal.
Subsequent receivers block or take the wrong branch. Replace with a
closed chan struct{} which can be received from any number of times,
and store the error in a separate field.
2026-03-19 17:44:28 -07:00
Jesse Gross 0ff7d724ff mlx: fix subprocess log deadlock
The stderr reader used bufio.Scanner which has a 64KB max line size.
If the subprocess wrote a line exceeding this limit, the scanner would
stop reading, the OS pipe buffer would fill, and the subprocess would
deadlock.

Replace the scanner with a statusWriter that wraps io.Copy. The writer
forwards all stderr to os.Stderr while capturing the last short line
(≤256 bytes) for error reporting, avoiding both the deadlock and the
need to buffer arbitrarily long lines.
2026-03-19 17:44:28 -07:00
Devon Rifkin 46cb7795e1 add ability to turn on debug request logging (#14106)
If `OLLAMA_DEBUG_LOG_REQUESTS` is set, then on server startup a temp
folder will be created. Upon any inference request, the body will be
logged to a file in this folder, as well as a small shell script to
"replay" the request using cURL.

This is just intended for debugging scenarios, not as something to turn
on normally.
2026-03-19 17:08:17 -07:00
Bruce MacDonaldandDongluo Chen 126d8db7f3 parsers: robust xml tool repair (#14961)
Previous xml repair for glm was a good start, but we need to go further and repair any incorrect open or closing tags

Co-authored-by: Dongluo Chen <dongluo.chen@gmail.com>
2026-03-19 11:24:48 -07:00
Eva H 3f3a24b418 app: fix desktop app stuck loading when OLLAMA_HOST is an unspecified bind address (#14885) 2026-03-19 12:57:57 -04:00
Jesse Gross 96e36c0d90 mlxrunner: share KV cache across conversations with common prefixes
Enable multiple conversations to reuse cached computations when they
share token prefixes (e.g. the same system prompt). A prefix trie
tracks shared regions so switching between conversations only
recomputes tokens that diverge. Inactive conversation state is paged
from active GPU memory to other memory and restored on demand, with LRU
eviction to keep memory usage bounded.
2026-03-18 16:06:33 -07:00
Jesse Gross 6f8ddbb26b mlxrunner: fix Slice(0, 0) returning full dimension instead of empty
Slice used cmp.Or to resolve a zero stop value to the dimension size,
intended to support open-ended slices like a[i:]. This made Slice(0, 0)
indistinguishable from Slice(), so any slice with a zero stop would
silently include the entire dimension instead of being empty.

Replace cmp.Or with an explicit End sentinel and resolve negative
indices against the dimension size, matching Python/PyTorch semantics.
2026-03-18 16:06:33 -07:00
Eva H b5e7888414 cmd/launch: skip redundant config writes when model unchanged (#14941) 2026-03-18 17:36:52 -04:00
Parth Sareen eab4d22269 docs: update claude code and openclaw for web search (#14922) 2026-03-18 14:18:49 -07:00
Bruce MacDonald 5759c2d2d2 launch: fix openclaw not picking up newly selected model (#14943)
Sessions with a stale model field were not updated when the primary
changed, so the old model continued to be used.
2026-03-18 13:20:10 -07:00
Bruce MacDonald 42b1c2642b docs: update minimax-m2.5 references to m2.7 (#14942) 2026-03-18 12:59:28 -07:00
Bruce MacDonald 727d69ddf3 tui: fix signin on headless Linux systems (#14627)
Defensively handle environments without a display server to ensure signin remains usable on headless VMs and SSH sessions.

- Skip calling xdg-open when neither DISPLAY nor WAYLAND_DISPLAY is set, preventing silent failures or unexpected browser handlers
- Render the signin URL as plain text instead of wrapping it in OSC 8 hyperlink escape sequences, which can be garbled or hidden by terminals that don't support them
2026-03-18 11:11:17 -07:00
Jesse Gross f622b0c5fc launch: disable claude attribution header to preserve KV cache
Claude Code sends an x-anthropic-billing-header that changes on every
request. This is embedded in the system prompt and consequently
breaks the KV cache for every request. Given the size of the prompts
that Claude Code usees, this has significant performance impact.
2026-03-17 20:48:03 -07:00
Bruce MacDonald 5d0000634c cmd/launch: check for both npm and git before installing OpenClaw (#14888)
The OpenClaw installer requires git in addition to npm. Update the
dependency check to detect both and provide specific install guidance
for whichever dependencies are missing.
2026-03-17 18:20:05 -07:00
Parth Sareen 676d9845ba launch: register websearch for openclaw (#14914) 2026-03-17 15:03:15 -07:00
Devon Rifkin e37a9b4c01 cloud_proxy: for the web_search legacy path, flush on newlines (#14897)
`WebSearchAnthropicWriter` expects a single object per write. The new
transparent proxy will instead send it whatever bytes it sees. This
cloud-model + local-orchestration + cloud-search is a temporary code
path, so instead of making the web search code more robust to this, I
put an adapter in the middle that will flush line-by-line to preserve
the old behavior.
2026-03-17 13:30:17 -07:00
Patrick Devine d727aacd04 mlx: quantized embeddings, fast SwiGLU, and runtime fixes (#14884)
Add QuantizedEmbedding and EmbeddingLayer interface so models can
use quantized embedding weights and expose tied output projections.
This change updates gemma3, glm4_moe_lite, llama, qwen3, and qwen3_5
to use the new interface.
2026-03-17 11:21:38 -07:00
Patrick Devine fa69b833cd mlx: add prequantized tensor packing + changes for qwen35 (#14878)
This change adds a tensorImportTransform interface for model-specific
tensor transformations during safetensors import. This allows importing
and modifying the standard HF based weights as well as the mlx-community
derived pre-quantized safetensors repos to be directly
imported into `ollama create`. Right now this only works with Qwen3.5
importing which does tensor renaming, norm weight shifting (it
adds +1 to each value of the norm vectors), conv1d transposition,
and casts to BF16s for F32 based vectors.
2026-03-17 11:21:18 -07:00
Jesse Gross bbbad97686 sched: Model eviction for MLX
MLX runners (image generation and LLM) previously bypassed the
scheduler's standard load path via a separate loadMLX method. This meant
they skipped VRAM fitting checks and couldn't participate in model
eviction.

Now all model types flow through the same load function. Model eviction
for MLX is based on weights as KV cache and compute graph are dynamic.
This means that eviction does not take into account the worst case
memory and models can still compete for memory but it is a significant
improvement.
2026-03-16 17:40:29 -07:00
Parth Sareen bcf6d55b54 launch: fix web search, add web fetch, and enable both for local (#14886) 2026-03-16 16:26:19 -07:00
easonysliuandClaude 810d4f9c22 runner: fix swallowed error in allocModel graph reservation
In allocModel(), the first call to reserveWorstCaseGraph(true) had its
error silently discarded — `return nil` was used instead of `return err`.

This meant that if the prompt-sized graph reservation failed (e.g. due
to insufficient memory), the error was swallowed, allocModel reported
success, and the model appeared to load correctly. Subsequent inference
would then fail in unexpected ways because the worst-case graph was
never properly reserved.

Fix: return the actual error so the caller can handle the failure
(retry with reduced parallelism, report OOM, etc.).

Co-Authored-By: Claude (claude-opus-4-6) <noreply@anthropic.com>
2026-03-16 15:48:45 -07:00
Bruce MacDonald 856c047a6c cmd/launch: skip --install-daemon when systemd is unavailable (#14883)
In container environments without systemd, `openclaw onboard
--install-daemon` exits non-zero because it cannot create a systemd
user service. This causes `ollama launch openclaw` to abort even
though the gateway can be started as a foreground child process.

Only pass --install-daemon when systemd user services are reachable
(Linux with /run/systemd/system present and XDG_RUNTIME_DIR set).
On all other platforms the flag is still included by default.
2026-03-16 13:50:04 -07:00
Daniel Hiltgen 79c1e93c00 bench: improve benchmarking tool (#14240)
New features:
- Warmup phase to eliminate cold-start outliers
- time-to-first-token measured in each epoch
- VRAM/memory tracking to identify CPU spillover
- Controlled prompt length
- Defaults to 6 epochs and 200 tokens max

Benchstat fixes:
- ns/request instead of ns/op — non-standard unit created a separate group instead of grouping with timing metrics
- Token count as the N field — benchstat interprets N as iteration count for statistical weighting, not as a token count
2026-03-15 11:47:31 -07:00
Parth Sareen f8b657c967 cmd/launch: add guards for headless mode (#14837) 2026-03-14 00:10:02 -07:00
Bruce MacDonald 10fefe0d57 config: use native OpenClaw Ollama onboarding (#14829)
OpenClaw now accepts the Ollama onboarding flags directly upstream, so rely on its wizard state instead of the legacy integration onboarding flag.

Update first-run setup to pass the Ollama auth and model flags during onboarding, perform a best-effort update before onboarding when needed, and drop the stale test that asserted persistence of the old onboarding flag.
2026-03-13 16:28:40 -07:00
Daniel Hiltgen 2f9a68f9e9 rocm: doc driver constraints (#14833) 2026-03-13 15:53:35 -07:00
Bruce MacDonald 3980c0217d server: decompress zstd request bodies in cloud passthrough middleware (#14827)
When a zstd-compressed request (e.g. from Codex CLI) hits /v1/responses
with a cloud model the request failed.

Fix by decompressing zstd bodies before
model extraction, so cloud models are detected and proxied directly
without the writer being wrapped.
2026-03-13 15:06:47 -07:00
Parth Sareen 870599f5da launch: remove warning for default policy (#14830) 2026-03-13 15:01:38 -07:00
Bruce MacDonald abf8e8e9c8 middleware: handle non-JSON error responses gracefully (#14828)
writeError in both OpenAI and Anthropic middleware writers would return
a raw json.SyntaxError when the error payload wasn't valid JSON (e.g.
"invalid character 'e' looking for beginning of value"). Fall back to
using the raw bytes as the error message instead.

Also use the actual HTTP status code rather than hardcoding 500, so
error types map correctly
2026-03-13 14:50:49 -07:00
Shivam Tiwari f3f31a8192 anthropic: close thinking block before tool_use when no text in between (#14825)
Root cause: StreamConverter.Process() only incremented contentIndex when
closing a thinking block if text content was present. When a model emitted
thinking followed directly by a tool_use block (no text in between),
thinkingDone was never set and contentIndex was not incremented, causing the
tool_use content_block_start to reuse index 0. Clients expecting sequential
indices would then fail to find the tool content block.

Fix: In the tool call loop, close any open thinking block (thinkingStarted &&
!thinkingDone) and increment contentIndex before opening the tool_use block,
mirroring the existing logic that closes an open text block.

Fixes #14816
2026-03-13 13:12:05 -07:00
Devon Rifkin 9e7ba835da cmd: still populate ollama ls when using ollama run <model:cloud> (#14824)
This is temporary until `api/tags` supports cloud natively
2026-03-13 12:24:45 -07:00
Parth Sareen 347f17b8d1 launch: add compact window for claude code (#14823) 2026-03-13 12:09:23 -07:00
Devon Rifkin 081b9eb423 api/create: always propagate :cloud source for cloud models (#14822)
Otherwise, using `/save` would try to run the local model instead
2026-03-13 11:58:00 -07:00
Parth Sareen bb867c6fdb launch: fix headless --yes integration flow and policy scoping (#14815) 2026-03-13 11:45:36 -07:00
Cadu 81f4506a61 docs: document reasoning_effort support in OpenAI-compatible API (#14821)
Add reasoning_effort and reasoning to the supported features and
request fields for /v1/chat/completions. These fields control
thinking on thinking-capable models but were previously undocumented.

Closes #14820
2026-03-13 10:57:14 -07:00
Parth Sareen 76925f1284 cmd: TUI model ordering (#14814) 2026-03-13 10:19:22 -07:00
Devon Rifkin f676231de9 server: remove experimental aliases support (#14810) 2026-03-12 20:27:24 -07:00
Parth Sareen af5f7c0a9e cmd: refactor tui and launch (#14609) 2026-03-12 18:39:06 -07:00
Daniel Hiltgen a6b27d776b ci: fix missing windows zip file (#14807)
Use 7z compression (better compression rate) if found in path.  That
alone isn't sufficient to get us under 2G, so MLX is now split out as a
discrete download.  Fix CI so it will fail if artifacts fail to upload.
2026-03-12 16:14:00 -07:00
Daniel Hiltgen 539741199e mlx: perf improvements (#14768)
* mlx: perf improvements

Fix nn.go to call mlx_fast_layer_norm instead of manually implementing (mean,
subtract, variance, rsqrt, multiply, add — 6 ops)

Fix llama.go, gemma3.go to remove RepeatKV to tile K/V tensors to match the Q
head count, since scaled_dot_product_attention natively handles GQA (it just
requires n_q_heads % n_kv_heads == 0)

* review comments
2026-03-12 12:01:28 -07:00
Eva H 8f45236d09 middleware: enable local tool model for web search (#14787) 2026-03-11 17:51:39 -04:00
Parth Sareen 97013a190c openai: split mixed thinking stream chunks via ToChunks (#14648) 2026-03-11 14:21:29 -07:00
Daniel Hiltgen c222735c02 mlx: only log load errors when MLX is needed (#14764)
This suppresses irrelevant/noisy errors in the GGML runner.
2026-03-11 10:31:31 -07:00
Daniel Hiltgen 87d21c7fc0 MLX: harden for init failures (#14777)
The CLI now links to the lazy-load MLX code, but that still happens in
init functions.  On internal MLX errors, the CLI exits before it has a
chance to start.  This change re-wires the MLX error handling so it
doesn't exit by default.  The MLX based runners currently expect exits
on failure, so they re-initialize the default error handling.  We can
refine error handling for better go stack traces in the future.
2026-03-10 22:52:23 -07:00
Jeffrey Morgan 54e05172a0 Revert "runner: add token history sampling parameters to ollama runner (#14537)" (#14776)
This reverts commit 86513cb697.
2026-03-10 21:07:52 -07:00
Parth Sareen 464186e995 config: qwen3.5 recommendations (#14758) 2026-03-10 18:04:57 -07:00
Devon Rifkin 8c4d5d6c2f cloud_proxy: send ollama client version (#14769)
This was previously included in the user agent, and we've made use of it
in the past to hotpatch bugs server-side for particular Ollama versions.
2026-03-10 15:53:25 -07:00
Parth Sareen bc72b14016 docs: update claude code docs (#14770) 2026-03-10 15:52:41 -07:00
Parth Sareen 61086083eb server: add experimental web search and web fetch routes (#14753) 2026-03-09 21:52:12 -07:00
Daniel Hiltgen 62d1f01ab4 ci: Fix windows build (#14754)
Instead of relying on sh for wildcard, do it in Go for better windows
compatibility.
2026-03-09 19:27:59 -07:00
Daniel Hiltgen 10e51c5177 MLX: add header vendoring and remove go build tag (#14642)
* prefer rocm v6 on windows

Avoid building with v7 - more changes are needed

* MLX: add header vendoring and remove go build tag

This switches to using a vendoring approach for the mlx-c headers so that Go
can build without requiring a cmake first.  This enables building the new MLX
based code by default.  Every time cmake runs, the headers are refreshed, so we
can easily keep them in sync when we bump mlx versions.  Basic Windows
and Linux support are verified.

* ci: harden for flaky choco repo servers

CI sometimes fails due to choco not actually installing cache.  Since it just speeds up the build, we can proceed without.

* review comments
2026-03-09 17:24:45 -07:00
Patrick Devine 3e06bde643 mlx: get parameters from modelfile during model creation (#14747) 2026-03-09 15:33:24 -07:00
Eva H 6be2de8214 app: auto update should be enabled when reset to defaults (#14741) 2026-03-09 15:02:36 -04:00
Daniel Hiltgen ebb1b9ec14 rocm: update linux to v7.2 (#14391)
* rocm: update linux to v7.2

* review comments
2026-03-09 08:26:55 -07:00
Patrick Devine d126467d5d x/mlxrunner: replace sampler interface chain with single stateful Sampler (#14652)
- Collapse MLX sampling state into a single sample.Sampler struct (options + history).
- Replace interface-based sampler chain (TopP, TopK, penalty, etc.) with function-based transforms.
- Update request/pipeline wiring to use *sample.Sampler, seed history from prompt tokens, and append generated tokens each step.
- Implement top_p, min_p, repeat_penalty, and frequency_penalty
2026-03-07 17:50:57 -08:00
Devon Rifkin afb4c62fbf cloud_proxy: handle stream disconnects gracefully (#14685)
Previously we were printing out bad errors for expected cases like
clients disconnecting. Now we only debug log when that happens (which
still might help in cases where we're figuring out why an integration
isn't working). For other errors, we print out a proper warning now
2026-03-06 19:18:52 -08:00
Patrick Devine e790dc435b mlx: int4 groupsize 64 (#14682)
Change affine 4bit integers to use groupsize 64
2026-03-06 16:39:47 -08:00
Daniel Hiltgen 288077c3a3 build: smarter docker parallelism (#14653)
Our Dockerfile leverages parallel stages for more efficient builds.  However,
our old parallel settings were naive and lead to under/over utilization
depending on the capabilities of your build system.

This change switches to using Ninja for all our docker cmake builds to leverage
its smarter parallel logic.  We tell Ninja to target a load of nproc so each of
the build stages will share the load on the system aiming for full CPU use
without oversaturation.

The GPU parallelism settings are also adjusted to 4 to avoid a long-tail for
the last few GPU targets as they work through the long list of GPU
architectures.

This also fixes the Dockerfile to move Vulkan install to just the stage that
needs it instead of blocking most other GPU installs.  This should speed up CI
which always has a clean build cache.
2026-03-06 16:36:22 -08:00
Daniel Hiltgen 4425c54eda create: fix localhost handling (#14681) 2026-03-06 16:35:58 -08:00
Michael Yang 778899a5d2 docs: format compat docs (#14678) 2026-03-06 14:53:17 -08:00
Jeffrey MorganandParthSareen 4eab60c1e2 Reapply "don't require pulling stubs for cloud models" again (#14608)
* Revert "Revert "Reapply "don't require pulling stubs for cloud models"" (#14606)"

This reverts commit 39982a954e.

* fix test + do cloud lookup only when seeing cloud models

---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-03-06 14:27:47 -08:00
Bruce MacDonald 1af850e6e3 parsers: repair unclosed arg_value tags in GLM tool calls (#14656)
GLM models sometimes omits </arg_value> closing tags in tool call XML, causing xml.Unmarshal to fail with "element <arg_value> closed by </tool_call>".

This is a known issue across the GLM family.

Sanitize the input to fix closing arg_key values so encoding/xml can handle it.
2026-03-06 14:08:34 -08:00
Parth Sareen 9b0c7cc7b9 cmd: override stale entries for context window pi (#14655) 2026-03-05 16:30:24 -08:00
Daniel Hiltgen 6928630601 mlx: prevent remote creation mismatch (#14651)
If the user is pointing at a remote OLLAMA_HOST, fail experimental safetensor
based create operations as we only support local creation currently.
2026-03-05 14:59:00 -08:00
Parth Sareen 9896e3627f cmd/config: fix cloud model limit lookups in integrations (#14650) 2026-03-05 13:57:28 -08:00
Bruce MacDonald 15732f0ea7 cmd: use native Ollama API endpoint for OpenClaw (#14649)
Remove the /v1 suffix from the OpenClaw provider baseUrl so it uses
the native Ollama API instead of the OpenAI-compatible endpoint. The
/v1 endpoint my break tool calling in OpenClaw.
2026-03-05 13:29:17 -08:00
Parth Sareen 562c76d7cc cmd: add qwen3.5 context length for launch (#14626) 2026-03-04 14:10:52 -08:00
Parth Sareen 122c68c151 server: loosen thinking level constraint (#14625) 2026-03-04 13:42:18 -08:00
Jeffrey Morgan 82848a7806 model: fix renderer and parser for qwen3.5 (#14605) 2026-03-03 20:58:29 -08:00
Jeffrey Morgan 39982a954e Revert "Reapply "don't require pulling stubs for cloud models"" (#14606)
This reverts commit 799e51d419.
2026-03-03 20:56:10 -08:00
Patrick Devine e9f6ea232f Add qwen3.5-next-moe support to MLX runner and models (#14417)
This change adds support for qwen3.5-next-moe models (qwen3-next/qwen3.5-next/qwen3-coder) to the MLX runner. It also:

* introduces recurrent cache support and related MLX ops
* updates pipeline/runner integration and adds tests
* properly quantizes stacked expert tensors
* a Gated Delta Metal kernel for fast SSM inference
* adds new MLX calls for Conv1d, DepthwideConv1d, Contiguous, Exp, Log, SoftmaxAxis
2026-03-03 16:39:22 -08:00
Patrick Devine 110eff01a9 chore: remove old imagegen LLMs models (#14597)
These models are implemented in the x/mlxrunner instead.
2026-03-03 13:23:40 -08:00
Jeffrey Morgan 799e51d419 Reapply "don't require pulling stubs for cloud models"
This reverts commit 97d2f05a6d.
2026-03-03 13:17:10 -08:00
Victor-Quqi e8fcb29586 model/renderers: fix glm-ocr image tags in renderer prompts (#14584) 2026-03-03 12:51:34 -08:00
Jeffrey Morgan 97d2f05a6d Revert "don't require pulling stubs for cloud models (#14574)" (#14596)
This reverts commit 8207e55ec7.
2026-03-03 12:51:23 -08:00
Devon RifkinandParth Sareen 8207e55ec7 don't require pulling stubs for cloud models (#14574)
* don't require pulling stubs for cloud models

This is a first in a series of PRs that will better integrate Ollama's
cloud into the API and CLI. Previously we used to have a layer of
indirection where you'd first have to pull a "stub" model that contains
a reference to a cloud model. With this change, you don't have to pull
first, you can just use a cloud model in various routes like `/api/chat`
and `/api/show`. This change respects
<https://github.com/ollama/ollama/pull/14221>, so if cloud is disabled,
these models won't be accessible.

There's also a new, simpler pass-through proxy that doesn't convert the
requests ahead of hitting the cloud models, which they themselves
already support various formats (e.g., `v1/chat/completions` or Open
Responses, etc.). This will help prevent issues caused by double
converting (e.g., `v1/chat/completions` converted to `api/chat` on the
client, then calling cloud and converting back to a
`v1/chat/completions` response instead of the cloud model handling the
original `v1/chat/completions` request first).

There's now a notion of "source tags", which can be mixed with existing
tags. So instead of having different formats like`gpt-oss:20b-cloud` vs.
`kimi-k2.5:cloud` (`-cloud` suffix vs. `:cloud`), you can now specify
cloud by simply appending `:cloud`. This PR doesn't change model
resolution yet, but sets us up to allow for things like omitting the
non-source tag, which would make something like `ollama run
gpt-oss:cloud` work the same way that `ollama run gpt-oss` already works
today.

More detailed changes:

- Added a shared model selector parser in `types/modelselector`:
  - supports `:cloud` and `:local`
  - accepts source tags in any position
  - supports legacy `:<tag>-cloud`
  - rejects conflicting source tags
- Integrated selector handling across server inference/show routes:
  - `GenerateHandler`, `ChatHandler`, `EmbedHandler`,
    `EmbeddingsHandler`, `ShowHandler`
- Added explicit-cloud passthrough proxy for ollama.com:
  - same-endpoint forwarding for `/api/*`, `/v1/*`, and `/v1/messages`
  - normalizes `model` (and `name` for `/api/show`) before forwarding
  - forwards request headers except hop-by-hop/proxy-managed headers
  - uses bounded response-header timeout
  - handles auth failures in a friendly way
- Preserved cloud-disable behavior (`OLLAMA_NO_CLOUD`)
- Updated create flow to support `FROM ...:cloud` model sources (though
  this flow uses the legacy proxy still, supporting Modelfile overrides
  is more complicated with the direct proxy approach)
- Updated CLI/TUI/config cloud detection to use shared selector logic
- Updated CLI preflight behavior so explicit cloud requests do not
  auto-pull local stubs

What's next?

- Cloud discovery/listing and cache-backed `ollama ls` / `/api/tags`
- Modelfile overlay support for virtual cloud models on OpenAI/Anthropic
  request families
- Recommender/default-selection behavior for ambiguous model families
- Fully remove the legacy flow

Fixes: https://github.com/ollama/ollama/issues/13801

* consolidate pull logic into confirmAndPull helper

pullIfNeeded and ShowOrPull shared identical confirm-and-pull logic.
Extract confirmAndPull to eliminate the duplication.

* skip local existence checks for cloud models

ModelExists and the TUI's modelExists both check the local model list,
which causes cloud models to appear missing. Return true early for
explicit cloud models so the TUI displays them beside the integration
name and skips re-prompting the model picker on relaunch.

* support optionally pulling stubs for newly-style names

We now normalize names like `<family>:<size>:cloud` into legacy-style
names like `<family>:<size>-cloud` for pulling and deleting (this also
supports stripping `:local`). Support for pulling cloud models is
temporary, once we integrate properly into `/api/tags` we won't need
this anymore.

* Fix server alias syncing

* Update cmd/cmd.go

Co-authored-by: Parth Sareen <parth.sareen@ollama.com>

* address comments

* improve some naming

---------

Co-authored-by: ParthSareen <parth.sareen@ollama.com>
2026-03-03 10:46:33 -08:00
Jesse Gross ad16bffc7d mlx: Remove peak memory from the API
This is still in flux so it is better to just log it for now.
2026-03-02 15:56:18 -08:00
Jesse Gross c1e3ef4bcc mlxrunner: Refcount pinned tensors
Otherwise, it is error prone to manage multiple components working
with the same tensor.
2026-03-02 15:56:06 -08:00
Parth Sareen a3093cd5e5 cmd/opencode: rename provider from "Ollama (local)" to "Ollama" (#14566)
The "(local)" qualifier is unnecessary since there's only one Ollama
provider. Existing configs with the old name are migrated automatically;
custom names are left unchanged.
2026-03-02 14:17:18 -08:00
Bruce MacDonald 23d4cad1a2 server: verify digest is not empty on create (#14555)
An empty digest is not a valid digest for an incoming create request. Reject empty digests at the api level.
2026-03-02 13:43:35 -08:00
Jeffrey Morgan 86513cb697 runner: add token history sampling parameters to ollama runner (#14537) 2026-03-01 19:16:07 -08:00
Jeffrey MorganandYossi Ovadia 3490e9590b model/qwen3next: avoid crash in in DeltaNet when offloading (#14541)
Co-authored-by: Yossi Ovadia <jabadia@gmail.com>
2026-03-01 18:44:04 -08:00
Jeffrey Morgan 8da09b1e7e qwen3next: add compatibility with imported GGUF models (#14517) 2026-02-28 14:21:42 -08:00
Jesse Gross a60b9adcce mlxrunner: Fix prompt eval timing and count metrics
Only the last token's processing time is included in prompt processing,
giving an artificially high rate. In addition, the number of tokens
only included the tokens that miss the cache, instead of our historic
total tokens.
2026-02-27 17:29:47 -08:00
Jesse Gross a16f96658b mlxrunner: Enforce model context limit
Currently, context length is unbounded - the cache will keep
growing forever independent of the model's trained context
length. This caps it and enforces semantics similar to most
cloud services:
 - Long prompts will result in an error, not truncation.
 - Generation that exceeds the context will be stopped
2026-02-27 17:29:47 -08:00
Jesse Gross 18ab09b431 mlxrunner: Propagate pipeline errors to client via api.StatusError
Errors that occur during pipeline processing are currently only
logged but not sent back to the client. Rather than using HTTP
status codes as we have historically done, this serializes errors
as messages to allow sending them at any time during the stream.
2026-02-27 17:29:47 -08:00
Jesse Gross 638faeac54 mlxrunner: Report actual memory usage from runner
The MLX runner previously reported a static VRAM estimate that was
computed at load time and consisted only of the weights. This is
strictly less than the actual memory usage, as it does not include
the KV cache or compute graph.
2026-02-27 17:29:47 -08:00
Jesse Gross dd5eb6337d mlxrunner: Fix panic on full KV cache hit
When the entire prompt was already cached (e.g. repeated prompt),
findRemaining returned an empty slice, causing FromValues to panic
on an index-out-of-range accessing a zero-length byte slice.

Fix by always keeping at least one token to re-evaluate so the
pipeline can seed token generation. Also reject empty prompts
early rather than panicking.
2026-02-27 11:07:03 -08:00
Patrick Devine 79917cf80b show peak memory usage (#14485) 2026-02-26 18:38:27 -08:00
Parth Sareen cc90a035a0 model/parsers: add stable tool call indexing for glm47 and qwen3 parsers (#14484) 2026-02-26 18:14:29 -08:00
1413 changed files with 101812 additions and 438494 deletions

No files matched your search

+133 -27
View File
@@ -24,10 +24,10 @@ jobs:
run: |
echo GOFLAGS="'-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=${GITHUB_REF_NAME#v}\" \"-X=github.com/ollama/ollama/server.mode=release\"'" | tee -a $GITHUB_OUTPUT
echo VERSION="${GITHUB_REF_NAME#v}" | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo vendorsha=$(cat LLAMA_CPP_VERSION) | tee -a $GITHUB_OUTPUT
darwin-build:
runs-on: macos-14-xlarge
runs-on: macos-26-xlarge
environment: release
needs: setup-environment
env:
@@ -57,7 +57,7 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
- run: |
./scripts/build_darwin.sh
- name: Log build results
@@ -106,10 +106,10 @@ jobs:
flags: ''
- os: windows
arch: amd64
preset: 'ROCm 6'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
rocm-version: '6.2'
flags: '-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
preset: 'ROCm 7'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
flags: '-DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
runner_dir: 'rocm'
- os: windows
arch: amd64
@@ -117,6 +117,25 @@ jobs:
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
flags: ''
runner_dir: 'vulkan'
- os: windows
arch: amd64
preset: 'MLX CUDA 13'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
- '"cufft"'
- '"cufft_dev"'
- '"nvrtc"'
- '"nvrtc_dev"'
- '"crt"'
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
flags: ''
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
env:
@@ -125,8 +144,10 @@ jobs:
- name: Install system dependencies
run: |
choco install -y --no-progress ccache ninja
ccache -o cache_dir=${{ github.workspace }}\.ccache
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan')
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'ROCm ') || startsWith(matrix.preset, 'Vulkan') || startsWith(matrix.preset, 'MLX ')
id: cache-install
uses: actions/cache/restore@v4
with:
@@ -134,8 +155,9 @@ jobs:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- if: startsWith(matrix.preset, 'CUDA ')
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- if: startsWith(matrix.preset, 'CUDA ') || startsWith(matrix.preset, 'MLX ')
name: Install CUDA ${{ matrix.cuda-version }}
run: |
$ErrorActionPreference = "Stop"
@@ -179,6 +201,23 @@ jobs:
run: |
echo "CC=clang.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CXX=clang++.exe" | Out-File -FilePath $env:GITHUB_ENV -Append
- if: startsWith(matrix.preset, 'MLX ')
name: Install cuDNN for MLX
run: |
$ErrorActionPreference = "Stop"
$cudnnRoot = "C:\Program Files\NVIDIA\CUDNN"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.cudnn-install }}" -OutFile "cudnn.zip"
Expand-Archive -Path cudnn.zip -DestinationPath cudnn-extracted
$cudnnDir = (Get-ChildItem -Path cudnn-extracted -Directory)[0].FullName
New-Item -ItemType Directory -Force -Path $cudnnRoot
Copy-Item -Path "$cudnnDir\*" -Destination "$cudnnRoot\" -Recurse
}
echo "CUDNN_ROOT_DIR=$cudnnRoot" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CUDNN_INCLUDE_PATH=$cudnnRoot\include" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CUDNN_LIBRARY_PATH=$cudnnRoot\lib\x64" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "$cudnnRoot\bin\x64" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- if: ${{ !cancelled() && steps.cache-install.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v4
with:
@@ -186,20 +225,77 @@ jobs:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.preset }}-${{ needs.setup-environment.outputs.vendorsha }}
- name: Build target "${{ matrix.preset }}"
- name: Build MLX
if: startsWith(matrix.preset, 'MLX ')
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }} --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build --parallel ([Environment]::ProcessorCount) --preset "${{ matrix.preset }}"
cmake --install build --component "${{ startsWith(matrix.preset, 'CUDA ') && 'CUDA' || startsWith(matrix.preset, 'ROCm ') && 'HIP' || startsWith(matrix.preset, 'Vulkan') && 'Vulkan' || 'CPU' }}" --strip
Remove-Item -Path dist\lib\ollama\rocm\rocblas\library\*gfx906* -ErrorAction SilentlyContinue
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }} "-DCMAKE_CUDA_FLAGS=-t 6" --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build --preset "${{ matrix.preset }}" -- -j 16
cmake --install build --component MLX --strip
env:
CMAKE_GENERATOR: Ninja
- name: Build llama-server
if: matrix.preset == 'CPU'
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake -S llama\server --preset cpu --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build build\llama-server-cpu -- -l $([Environment]::ProcessorCount)
cmake --install build\llama-server-cpu --component llama-server --strip
env:
CMAKE_GENERATOR: Ninja
- name: Build llama-server (CUDA)
if: startsWith(matrix.preset, 'CUDA ')
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
$cudaVer = "${{ matrix.preset }}".Replace('CUDA ', '').Replace(' ', '')
$preset = "cuda-v$cudaVer"
if ($cudaVer -eq "13") { $preset = "cuda-v13-windows" }
$buildDir = "build\llama-server-cuda-v$cudaVer"
cmake -S llama\server --preset $preset ${{ matrix.flags }} --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build $buildDir -- -l $([Environment]::ProcessorCount)
cmake --install $buildDir --component llama-server --strip
env:
CMAKE_GENERATOR: Ninja
- name: Build llama-server (ROCm)
if: startsWith(matrix.preset, 'ROCm ')
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
$hipPath = $env:CMAKE_PREFIX_PATH
$env:HIPCXX = "$hipPath\bin\clang++.exe"
$env:HIP_PLATFORM = "amd"
$env:CMAKE_PREFIX_PATH = "$hipPath"
$env:CC = "$hipPath\bin\clang.exe"
$env:CXX = "$hipPath\bin\clang++.exe"
cmake -S llama\server --preset rocm -G Ninja `
-DGGML_OPENMP=OFF `
-DCMAKE_HIP_FLAGS="-parallel-jobs=4" `
-DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" `
-DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" `
-DAMDGPU_TARGETS="gfx942;gfx950;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-" `
--install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build build\llama-server-rocm -- -l $([Environment]::ProcessorCount)
cmake --install build\llama-server-rocm --component llama-server --strip
env:
CMAKE_GENERATOR: Ninja
- name: Build llama-server (Vulkan)
if: matrix.preset == 'Vulkan'
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake -S llama\server --preset vulkan --install-prefix "$((pwd).Path)\dist\${{ matrix.os }}-${{ matrix.arch }}"
cmake --build build\llama-server-vulkan -- -l $([Environment]::ProcessorCount)
cmake --install build\llama-server-vulkan --component llama-server --strip
env:
CMAKE_GENERATOR: Ninja
- name: Log build results
@@ -257,7 +353,7 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
- name: Verify gcc is actually clang
run: |
$ErrorActionPreference='Continue'
@@ -313,7 +409,7 @@ jobs:
go-version-file: go.mod
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
- uses: actions/download-artifact@v4
with:
pattern: depends-windows*
@@ -347,9 +443,6 @@ jobs:
- os: linux
arch: amd64
target: archive
- os: linux
arch: amd64
target: rocm
- os: linux
arch: arm64
target: archive
@@ -381,9 +474,11 @@ jobs:
case "$COMPONENT" in
bin/ollama*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/*.so*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/llama-server*|lib/ollama/llama-quantize*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/cuda_v*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/vulkan*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}.tar.in ;;
lib/ollama/mlx*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/include*) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-mlx.tar.in ;;
lib/ollama/cuda_jetpack5) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack5.tar.in ;;
lib/ollama/cuda_jetpack6) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-jetpack6.tar.in ;;
lib/ollama/rocm) echo $COMPONENT >>ollama-${{ matrix.os }}-${{ matrix.arch }}-rocm.tar.in ;;
@@ -398,7 +493,7 @@ jobs:
done
- run: |
for ARCHIVE in dist/${{ matrix.os }}-${{ matrix.arch }}/*.tar.in; do
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd --ultra -22 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst);
tar c -C dist/${{ matrix.os }}-${{ matrix.arch }} -T $ARCHIVE --owner 0 --group 0 | zstd -9 -T0 >$(basename ${ARCHIVE//.*/}.tar.zst);
done
- uses: actions/upload-artifact@v4
with:
@@ -417,12 +512,14 @@ jobs:
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
- os: linux
arch: amd64
build-args: |
CGO_CFLAGS
CGO_CXXFLAGS
GOFLAGS
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
- os: linux
arch: amd64
suffix: '-rocm'
@@ -431,6 +528,7 @@ jobs:
CGO_CXXFLAGS
GOFLAGS
FLAVOR=rocm
APT_MIRROR=http://azure.archive.ubuntu.com/ubuntu
runs-on: ${{ matrix.arch == 'arm64' && format('{0}-{1}', matrix.os, matrix.arch) || matrix.os }}
environment: release
needs: setup-environment
@@ -543,11 +641,19 @@ jobs:
for payload in dist/*.txt dist/*.zip dist/*.tgz dist/*.tar.zst dist/*.exe dist/*.dmg dist/*.ps1 dist/*.sh ; do
echo "Uploading $payload"
gh release upload ${GITHUB_REF_NAME} $payload --clobber &
pids[$!]=$!
pids+=($!)
sleep 1
done
echo "Waiting for uploads to complete"
for pid in "${pids[*]}"; do
wait $pid
failed=0
for pid in "${pids[@]}"; do
if ! wait $pid; then
echo "::error::Upload failed (pid $pid)"
failed=1
fi
done
if [ $failed -ne 0 ]; then
echo "One or more uploads failed"
exit 1
fi
echo "done"
+107 -28
View File
@@ -37,8 +37,8 @@ jobs:
| xargs python3 -c "import sys; from pathlib import Path; print(any(Path(x).match(glob) for x in sys.argv[1:] for glob in '$*'.split(' ')))"
}
echo changed=$(changed 'llama/llama.cpp/**/*' 'ml/backend/ggml/ggml/**/*') | tee -a $GITHUB_OUTPUT
echo vendorsha=$(make -f Makefile.sync print-base) | tee -a $GITHUB_OUTPUT
echo changed=$(changed 'llama/server/**/*' 'LLAMA_CPP_VERSION' '.github/**/*') | tee -a $GITHUB_OUTPUT
echo vendorsha=$(cat LLAMA_CPP_VERSION) | tee -a $GITHUB_OUTPUT
linux:
needs: [changes]
@@ -47,19 +47,33 @@ jobs:
matrix:
include:
- preset: CPU
server_preset: cpu
server_build_dir: build/llama-server-cpu
- preset: CUDA
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
flags: '-DCMAKE_CUDA_ARCHITECTURES=87'
server_preset: cuda-v13
server_build_dir: build/llama-server-cuda-v13
- preset: ROCm
container: rocm/dev-ubuntu-22.04:6.1.2
container: rocm/dev-ubuntu-22.04:7.2.1
extra-packages: rocm-libs
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_PREFIX_PATH=/opt/rocm'
server_preset: rocm
server_build_dir: build/llama-server-rocm
- preset: Vulkan
container: ubuntu:22.04
extra-packages: >
mesa-vulkan-drivers vulkan-tools
libvulkan1 libvulkan-dev
vulkan-sdk cmake ccache g++ make
server_preset: vulkan
server_build_dir: build/llama-server-vulkan
- preset: 'MLX CUDA 13'
container: nvidia/cuda:13.0.0-devel-ubuntu22.04
extra-packages: libcudnn9-dev-cuda-13 libopenblas-dev liblapack-dev liblapacke-dev git curl
flags: '-DCMAKE_CUDA_ARCHITECTURES=87 -DBLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu -DLAPACK_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu'
install-go: true
mlx_preset: 'MLX CUDA 13'
runs-on: linux
container: ${{ matrix.container }}
steps:
@@ -75,20 +89,35 @@ jobs:
echo "deb [signed-by=/usr/share/keyrings/lunarg-archive-keyring.gpg] https://packages.lunarg.com/vulkan/1.4.313 jammy main" | $sudo tee /etc/apt/sources.list.d/lunarg-vulkan-1.4.313-jammy.list > /dev/null
$sudo apt-get update
fi
$sudo apt-get install -y cmake ccache ${{ matrix.extra-packages }}
$sudo apt-get install -y cmake ccache curl git ${{ matrix.extra-packages }}
# llama-server requires CMake 3.24+, while Ubuntu 22.04 ships 3.22.
curl -fsSL https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-linux-$(uname -m).tar.gz | $sudo tar xz -C /usr/local --strip-components 1
# Export VULKAN_SDK if provided by LunarG package (defensive)
if [ -d "/usr/lib/x86_64-linux-gnu/vulkan" ] && [ "${{ matrix.preset }}" = "Vulkan" ]; then
echo "VULKAN_SDK=/usr" >> $GITHUB_ENV
fi
env:
DEBIAN_FRONTEND: noninteractive
- if: matrix.install-go
name: Install Go
run: |
GO_VERSION=$(awk '/^go / { print $2 }' go.mod)
curl -fsSL "https://golang.org/dl/go${GO_VERSION}.linux-$(dpkg --print-architecture).tar.gz" | tar xz -C /usr/local
echo "/usr/local/go/bin" >> $GITHUB_PATH
- uses: actions/cache@v4
with:
path: /github/home/.cache/ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
cmake --preset ${{ matrix.preset }} ${{ matrix.flags }}
cmake --build --preset ${{ matrix.preset }} --parallel
- name: Build llama-server
if: matrix.server_preset
run: |
cmake -S llama/server --preset "${{ matrix.server_preset }}" ${{ matrix.flags }}
cmake --build "${{ matrix.server_build_dir }}" -- -l $(nproc)
- name: Build MLX
if: matrix.mlx_preset
run: |
cmake --preset "${{ matrix.mlx_preset }}" ${{ matrix.flags }}
cmake --build --preset "${{ matrix.mlx_preset }}" -- -l $(nproc)
windows:
needs: [changes]
@@ -97,9 +126,13 @@ jobs:
matrix:
include:
- preset: CPU
server_preset: cpu
server_build_dir: build\llama-server-cpu
- preset: CUDA
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
server_preset: cuda-v13-windows
server_build_dir: build\llama-server-cuda-v13
cuda-components:
- '"cudart"'
- '"nvcc"'
@@ -110,16 +143,41 @@ jobs:
- '"nvptxcompiler"'
cuda-version: '13.0'
- preset: ROCm
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-24.Q4-WinSvr2022-For-HIP.exe
flags: '-DAMDGPU_TARGETS=gfx1010 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
install: https://download.amd.com/developer/eula/rocm-hub/AMD-Software-PRO-Edition-26.Q1-Win11-For-HIP.exe
rocm-version: '7.1'
flags: '-DAMDGPU_TARGETS=gfx1010 -DGGML_OPENMP=OFF -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma" -DCMAKE_CXX_FLAGS="-parallel-jobs=4 -Wno-ignored-attributes -Wno-deprecated-pragma"'
server_preset: rocm
server_build_dir: build\llama-server-rocm
- preset: Vulkan
install: https://sdk.lunarg.com/sdk/download/1.4.321.1/windows/vulkansdk-windows-X64-1.4.321.1.exe
server_preset: vulkan
server_build_dir: build\llama-server-vulkan
- preset: 'MLX CUDA 13'
install: https://developer.download.nvidia.com/compute/cuda/13.0.0/local_installers/cuda_13.0.0_windows.exe
cudnn-install: https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/windows-x86_64/cudnn-windows-x86_64-9.18.1.3_cuda13-archive.zip
flags: '-DCMAKE_CUDA_ARCHITECTURES=80'
mlx_preset: 'MLX CUDA 13'
cuda-components:
- '"cudart"'
- '"nvcc"'
- '"cublas"'
- '"cublas_dev"'
- '"cufft"'
- '"cufft_dev"'
- '"nvrtc"'
- '"nvrtc_dev"'
- '"crt"'
- '"nvvm"'
- '"nvptxcompiler"'
cuda-version: '13.0'
runs-on: windows
steps:
- run: |
choco install -y --no-progress ccache ninja
ccache -o cache_dir=${{ github.workspace }}\.ccache
- if: matrix.preset == 'CUDA' || matrix.preset == 'ROCm' || matrix.preset == 'Vulkan'
if (Get-Command ccache -ErrorAction SilentlyContinue) {
ccache -o cache_dir=${{ github.workspace }}\.ccache
}
- if: matrix.preset == 'CUDA' || matrix.preset == 'ROCm' || matrix.preset == 'Vulkan' || matrix.preset == 'MLX CUDA 13'
id: cache-install
uses: actions/cache/restore@v4
with:
@@ -127,8 +185,9 @@ jobs:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
- if: matrix.preset == 'CUDA'
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- if: matrix.preset == 'CUDA' || matrix.preset == 'MLX CUDA 13'
name: Install CUDA ${{ matrix.cuda-version }}
run: |
$ErrorActionPreference = "Stop"
@@ -164,10 +223,27 @@ jobs:
Invoke-WebRequest -Uri "${{ matrix.install }}" -OutFile "install.exe"
Start-Process -FilePath .\install.exe -ArgumentList "-c","--am","--al","in" -NoNewWindow -Wait
}
$vulkanPath = (Resolve-Path "C:\VulkanSDK\*").path
echo "$vulkanPath\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "VULKAN_SDK=$vulkanPath" >> $env:GITHUB_ENV
- if: matrix.preset == 'MLX CUDA 13'
name: Install cuDNN for MLX
run: |
$ErrorActionPreference = "Stop"
$cudnnRoot = "C:\Program Files\NVIDIA\CUDNN"
if ("${{ steps.cache-install.outputs.cache-hit }}" -ne 'true') {
Invoke-WebRequest -Uri "${{ matrix.cudnn-install }}" -OutFile "cudnn.zip"
Expand-Archive -Path cudnn.zip -DestinationPath cudnn-extracted
$cudnnDir = (Get-ChildItem -Path cudnn-extracted -Directory)[0].FullName
New-Item -ItemType Directory -Force -Path $cudnnRoot
Copy-Item -Path "$cudnnDir\*" -Destination "$cudnnRoot\" -Recurse
}
echo "CUDNN_ROOT_DIR=$cudnnRoot" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CUDNN_INCLUDE_PATH=$cudnnRoot\include" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CUDNN_LIBRARY_PATH=$cudnnRoot\lib\x64" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "$cudnnRoot\bin\x64" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- if: ${{ !cancelled() && steps.cache-install.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v4
with:
@@ -175,17 +251,29 @@ jobs:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
C:\Program Files\AMD\ROCm
C:\VulkanSDK
key: ${{ matrix.install }}
C:\Program Files\NVIDIA\CUDNN
key: ${{ matrix.install }}-${{ matrix.cudnn-install }}
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: ${{ github.workspace }}\.ccache
key: ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.preset }}-${{ needs.changes.outputs.vendorsha }}
- run: |
- name: Build llama-server
if: matrix.server_preset
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.preset }}" ${{ matrix.flags }}
cmake --build --parallel --preset "${{ matrix.preset }}"
cmake -S llama\server --preset "${{ matrix.server_preset }}" ${{ matrix.flags }}
cmake --build "${{ matrix.server_build_dir }}" -- -l $([Environment]::ProcessorCount)
env:
CMAKE_GENERATOR: Ninja
- name: Build MLX
if: matrix.mlx_preset
run: |
Import-Module 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Microsoft.VisualStudio.DevShell.dll'
Enter-VsDevShell -VsInstallPath 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise' -SkipAutomaticLocation -DevCmdArguments '-arch=x64 -no_logo'
cmake --preset "${{ matrix.mlx_preset }}" ${{ matrix.flags }}
cmake --build --preset "${{ matrix.mlx_preset }}" -- -l $([Environment]::ProcessorCount)
env:
CMAKE_GENERATOR: Ninja
@@ -210,7 +298,7 @@ jobs:
go-version-file: 'go.mod'
cache-dependency-path: |
go.sum
Makefile.sync
LLAMA_CPP_VERSION
- uses: actions/setup-node@v4
with:
node-version: '20'
@@ -234,12 +322,3 @@ jobs:
- uses: golangci/golangci-lint-action@v9
with:
only-new-issues: true
patches:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Verify patches apply cleanly and do not change files
run: |
make -f Makefile.sync clean checkout apply-patches sync
git diff --compact-summary --exit-code
+1
View File
@@ -15,3 +15,4 @@ __debug_bin*
llama/build
llama/vendor
/ollama
integration/testdata/models/
+158 -140
View File
@@ -24,29 +24,19 @@ include(GNUInstallDirs)
find_package(Threads REQUIRED)
set(CMAKE_BUILD_TYPE Release)
set(BUILD_SHARED_LIBS ON)
# These defaults can be overridden by presets (e.g., for static macOS llama-server builds)
if(NOT DEFINED BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON)
endif()
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS ON) # Recent versions of MLX Requires gnu++17 extensions to compile properly
set(GGML_BUILD ON)
set(GGML_SHARED ON)
set(GGML_CCACHE ON)
set(GGML_BACKEND_DL ON)
set(GGML_BACKEND_SHARED ON)
set(GGML_SCHED_MAX_COPIES 4)
set(GGML_LLAMAFILE ON)
set(GGML_CUDA_PEER_MAX_BATCH_SIZE 128)
set(GGML_CUDA_GRAPHS ON)
set(GGML_CUDA_FA ON)
set(GGML_CUDA_COMPRESSION_MODE default)
if((CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_OSX_ARCHITECTURES MATCHES "arm64")
OR (NOT CMAKE_OSX_ARCHITECTURES AND NOT CMAKE_SYSTEM_PROCESSOR MATCHES "arm|aarch64|ARM64|ARMv[0-9]+"))
set(GGML_CPU_ALL_VARIANTS ON)
endif()
# GGML backend for inference is provided by llama-server (built separately via
# llama/server/CMakeLists.txt using FetchContent from upstream llama.cpp).
# This root CMakeLists.txt only builds MLX (when enabled).
if(APPLE)
set(CMAKE_BUILD_RPATH "@loader_path")
@@ -64,139 +54,62 @@ set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${OLLAMA_BUILD_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${OLLAMA_BUILD_DIR})
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/include)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cpu/amx)
add_compile_definitions(NDEBUG GGML_VERSION=0x0 GGML_COMMIT=0x0)
# Define GGML version variables for shared library SOVERSION
# These are required by ggml/src/CMakeLists.txt for proper library versioning
set(GGML_VERSION_MAJOR 0)
set(GGML_VERSION_MINOR 0)
set(GGML_VERSION_PATCH 0)
set(GGML_VERSION "${GGML_VERSION_MAJOR}.${GGML_VERSION_MINOR}.${GGML_VERSION_PATCH}")
set(GGML_CPU ON)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src)
set_property(TARGET ggml PROPERTY EXCLUDE_FROM_ALL TRUE)
get_target_property(CPU_VARIANTS ggml-cpu MANUALLY_ADDED_DEPENDENCIES)
if(NOT CPU_VARIANTS)
set(CPU_VARIANTS "ggml-cpu")
endif()
install(TARGETS ggml-base ${CPU_VARIANTS}
RUNTIME_DEPENDENCIES
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CPU
)
check_language(CUDA)
if(CMAKE_CUDA_COMPILER)
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.24" AND NOT CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES "native")
endif()
find_package(CUDAToolkit)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-cuda)
install(TARGETS ggml-cuda
RUNTIME_DEPENDENCIES
DIRECTORIES ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR}
PRE_INCLUDE_REGEXES cublas cublasLt cudart
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT CUDA
)
endif()
set(WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX "^gfx(908|90a|1200|1201):xnack[+-]$"
CACHE STRING
"Regular expression describing AMDGPU_TARGETS not supported on Windows. Override to force building these targets. Default \"^gfx(908|90a|1200|1201):xnack[+-]$\"."
)
check_language(HIP)
if(CMAKE_HIP_COMPILER)
set(HIP_PLATFORM "amd")
if(NOT AMDGPU_TARGETS)
find_package(hip REQUIRED)
list(FILTER AMDGPU_TARGETS INCLUDE REGEX "^gfx(94[012]|101[02]|1030|110[012]|120[01])$")
endif()
if(WIN32 AND WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX)
list(FILTER AMDGPU_TARGETS EXCLUDE REGEX ${WINDOWS_AMDGPU_TARGETS_EXCLUDE_REGEX})
endif()
if(AMDGPU_TARGETS)
find_package(hip REQUIRED)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-hip)
if (WIN32)
target_compile_definitions(ggml-hip PRIVATE GGML_CUDA_NO_PEER_COPY)
endif()
target_compile_definitions(ggml-hip PRIVATE GGML_HIP_NO_VMM)
install(TARGETS ggml-hip
RUNTIME_DEPENDENCY_SET rocm
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
install(RUNTIME_DEPENDENCY_SET rocm
DIRECTORIES ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR}
PRE_INCLUDE_REGEXES hipblas rocblas amdhip64 rocsolver amd_comgr hsa-runtime64 rocsparse tinfo rocprofiler-register drm drm_amdgpu numa elf
PRE_EXCLUDE_REGEXES ".*"
POST_EXCLUDE_REGEXES "system32"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP
)
foreach(HIP_LIB_BIN_INSTALL_DIR IN ITEMS ${HIP_BIN_INSTALL_DIR} ${HIP_LIB_INSTALL_DIR})
if(EXISTS ${HIP_LIB_BIN_INSTALL_DIR}/rocblas)
install(DIRECTORY ${HIP_LIB_BIN_INSTALL_DIR}/rocblas DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT HIP)
break()
endif()
endforeach()
endif()
endif()
if(NOT APPLE)
find_package(Vulkan)
if(Vulkan_FOUND)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/ml/backend/ggml/ggml/src/ggml-vulkan)
install(TARGETS ggml-vulkan
RUNTIME_DEPENDENCIES
PRE_INCLUDE_REGEXES vulkan
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT Vulkan
)
endif()
endif()
option(MLX_ENGINE "Enable MLX backend" OFF)
if(MLX_ENGINE)
if(MLX_CUDA_ARCHITECTURES OR CMAKE_CUDA_ARCHITECTURES)
check_language(CUDA)
endif()
message(STATUS "Setting up MLX (this takes a while...)")
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/x/imagegen/mlx)
# Find CUDA toolkit if MLX is built with CUDA support
find_package(CUDAToolkit)
# Build list of directories for runtime dependency resolution
set(MLX_RUNTIME_DIRS ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR})
# Add cuDNN bin paths for DLLs (Windows MLX CUDA builds)
# CUDNN_ROOT_DIR is the standard CMake variable for cuDNN location
if(DEFINED ENV{CUDNN_ROOT_DIR})
# cuDNN 9.x has versioned subdirectories under bin/ (e.g., bin/13.0/)
file(GLOB CUDNN_BIN_SUBDIRS "$ENV{CUDNN_ROOT_DIR}/bin/*")
list(APPEND MLX_RUNTIME_DIRS ${CUDNN_BIN_SUBDIRS})
endif()
# Add build output directory and MLX dependency build directories
list(APPEND MLX_RUNTIME_DIRS ${OLLAMA_BUILD_DIR})
# OpenBLAS DLL location (pre-built zip extracts into openblas-src/bin/)
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/openblas-src/bin)
# NCCL: on Linux, if real NCCL is found, cmake bundles libnccl.so via the
# regex below. If NCCL is not found, MLX links a static stub (OBJECT lib)
# so there is no runtime dependency. This path covers the stub build dir
# for windows so we include the DLL in our dependencies.
list(APPEND MLX_RUNTIME_DIRS ${CMAKE_BINARY_DIR}/_deps/mlx-build/mlx/distributed/nccl/nccl_stub-prefix/src/nccl_stub-build/Release)
# Base regexes for runtime dependencies (cross-platform)
set(MLX_INCLUDE_REGEXES cublas cublasLt cudart cufft nvrtc nvrtc-builtins cudnn nccl openblas gfortran)
# On Windows, also include dl.dll (dlfcn-win32 POSIX emulation layer)
if(WIN32)
list(APPEND MLX_INCLUDE_REGEXES "^dl\\.dll$")
endif()
install(TARGETS mlx mlxc
RUNTIME_DEPENDENCIES
DIRECTORIES ${CUDAToolkit_BIN_DIR} ${CUDAToolkit_BIN_DIR}/x64 ${CUDAToolkit_LIBRARY_DIR}
PRE_INCLUDE_REGEXES cublas cublasLt cudart nvrtc nvrtc-builtins cudnn nccl openblas gfortran
DIRECTORIES ${MLX_RUNTIME_DIRS}
PRE_INCLUDE_REGEXES ${MLX_INCLUDE_REGEXES}
PRE_EXCLUDE_REGEXES ".*"
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
if(TARGET jaccl)
install(TARGETS jaccl
RUNTIME DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
LIBRARY DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
FRAMEWORK DESTINATION ${OLLAMA_INSTALL_DIR} COMPONENT MLX
)
endif()
# Install the Metal library for macOS arm64 (must be colocated with the binary)
# Metal backend is only built for arm64, not x86_64
if(APPLE AND CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64")
@@ -205,15 +118,120 @@ if(MLX_ENGINE)
COMPONENT MLX)
endif()
# Manually install cudart and cublas since they might not be picked up as direct dependencies
# Install headers for NVRTC JIT compilation at runtime.
# MLX's own install rules use the default component so they get skipped by
# --component MLX. Headers are installed alongside libmlx in OLLAMA_INSTALL_DIR.
#
# Layout:
# ${OLLAMA_INSTALL_DIR}/include/cccl/{cuda,nv}/ — CCCL headers
# ${OLLAMA_INSTALL_DIR}/include/*.h — CUDA toolkit headers
#
# MLX's jit_module.cpp resolves CCCL via
# current_binary_dir()[.parent_path()] / "include" / "cccl"
# On Linux, MLX's jit_module.cpp resolves CCCL via
# current_binary_dir().parent_path() / "include" / "cccl", so we create a
# symlink from lib/ollama/include -> ${OLLAMA_RUNNER_DIR}/include
# This will need refinement if we add multiple CUDA versions for MLX in the future.
# CUDA runtime headers are found via CUDA_PATH env var (set by mlxrunner).
if(EXISTS ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/cuda
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
install(DIRECTORY ${CMAKE_BINARY_DIR}/_deps/cccl-src/include/nv
DESTINATION ${OLLAMA_INSTALL_DIR}/include/cccl
COMPONENT MLX)
endif()
# Install minimal CUDA toolkit headers needed by MLX JIT kernels.
# These are the transitive closure of includes from mlx/backend/cuda/device/*.cuh.
# The Go mlxrunner sets CUDA_PATH to OLLAMA_INSTALL_DIR so MLX finds them at
# $CUDA_PATH/include/*.h via NVRTC --include-path.
if(CUDAToolkit_FOUND)
file(GLOB CUDART_LIBS
# CUDAToolkit_INCLUDE_DIRS may be a semicolon-separated list
# (e.g. ".../include;.../include/cccl"). Find the entry that
# contains the CUDA runtime headers we need.
set(_cuda_inc "")
foreach(_dir ${CUDAToolkit_INCLUDE_DIRS})
if(EXISTS "${_dir}/cuda_runtime_api.h")
set(_cuda_inc "${_dir}")
break()
endif()
endforeach()
if(NOT _cuda_inc)
message(WARNING "Could not find cuda_runtime_api.h in CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
else()
set(_dst "${OLLAMA_INSTALL_DIR}/include")
set(_MLX_JIT_CUDA_HEADERS
builtin_types.h
cooperative_groups.h
cuda_bf16.h
cuda_bf16.hpp
cuda_device_runtime_api.h
cuda_fp16.h
cuda_fp16.hpp
cuda_fp8.h
cuda_fp8.hpp
cuda_runtime_api.h
device_types.h
driver_types.h
math_constants.h
surface_types.h
texture_types.h
vector_functions.h
vector_functions.hpp
vector_types.h
)
foreach(_hdr ${_MLX_JIT_CUDA_HEADERS})
install(FILES "${_cuda_inc}/${_hdr}"
DESTINATION ${_dst}
COMPONENT MLX)
endforeach()
# Subdirectory headers
install(DIRECTORY "${_cuda_inc}/cooperative_groups"
DESTINATION ${_dst}
COMPONENT MLX
FILES_MATCHING PATTERN "*.h")
install(FILES "${_cuda_inc}/crt/host_defines.h"
DESTINATION "${_dst}/crt"
COMPONENT MLX)
if(NOT WIN32 AND NOT APPLE)
install(CODE "
set(_link \"${CMAKE_INSTALL_PREFIX}/lib/ollama/include\")
set(_target \"${OLLAMA_RUNNER_DIR}/include\")
if(NOT EXISTS \${_link})
execute_process(COMMAND \${CMAKE_COMMAND} -E create_symlink \${_target} \${_link})
endif()
" COMPONENT MLX)
endif()
endif()
endif()
# On Windows, explicitly install dl.dll (dlfcn-win32 POSIX dlopen emulation)
# RUNTIME_DEPENDENCIES auto-excludes it via POST_EXCLUDE_FILES_STRICT because
# dlfcn-win32 is a known CMake target with its own install rules (which install
# to the wrong destination). We must install it explicitly here.
if(WIN32)
install(FILES ${OLLAMA_BUILD_DIR}/dl.dll
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
# Manually install CUDA runtime libraries that MLX loads via dlopen
# (not detected by RUNTIME_DEPENDENCIES since they aren't link-time deps)
if(CUDAToolkit_FOUND)
file(GLOB MLX_CUDA_LIBS
"${CUDAToolkit_LIBRARY_DIR}/libcudart.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*")
if(CUDART_LIBS)
install(FILES ${CUDART_LIBS}
"${CUDAToolkit_LIBRARY_DIR}/libcublas.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcublasLt.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc.so*"
"${CUDAToolkit_LIBRARY_DIR}/libnvrtc-builtins.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcufft.so*"
"${CUDAToolkit_LIBRARY_DIR}/libcudnn.so*")
if(MLX_CUDA_LIBS)
install(FILES ${MLX_CUDA_LIBS}
DESTINATION ${OLLAMA_INSTALL_DIR}
COMPONENT MLX)
endif()
endif()
endif()
+3 -126
View File
@@ -10,80 +10,6 @@
"CMAKE_MSVC_RUNTIME_LIBRARY": "MultiThreaded"
}
},
{
"name": "CPU",
"inherits": [ "Default" ]
},
{
"name": "CUDA",
"inherits": [ "Default" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50-virtual;60-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v11"
}
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50;52;60;61;70;75;80;86;89;90;90a;120",
"CMAKE_CUDA_FLAGS": "-Wno-deprecated-gpu-targets -t 2",
"OLLAMA_RUNNER_DIR": "cuda_v12"
}
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "75-virtual;80-virtual;86-virtual;87-virtual;89-virtual;90-virtual;90a-virtual;100-virtual;103-virtual;110-virtual;120-virtual;121-virtual",
"CMAKE_CUDA_FLAGS": "-t 4",
"OLLAMA_RUNNER_DIR": "cuda_v13"
}
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "72;87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack5"
}
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "87",
"OLLAMA_RUNNER_DIR": "cuda_jetpack6"
}
},
{
"name": "ROCm",
"inherits": [ "Default" ],
"cacheVariables": {
"CMAKE_HIP_PLATFORM": "amd"
}
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"cacheVariables": {
"CMAKE_HIP_FLAGS": "-parallel-jobs=4",
"AMDGPU_TARGETS": "gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-",
"OLLAMA_RUNNER_DIR": "rocm"
}
},
{
"name": "Vulkan",
"inherits": [ "Default" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "vulkan"
}
},
{
"name": "MLX",
"inherits": [ "Default" ],
@@ -94,15 +20,16 @@
},
{
"name": "MLX CUDA 12",
"inherits": [ "MLX", "CUDA 12" ],
"inherits": [ "MLX" ],
"cacheVariables": {
"OLLAMA_RUNNER_DIR": "mlx_cuda_v12"
}
},
{
"name": "MLX CUDA 13",
"inherits": [ "MLX", "CUDA 13" ],
"inherits": [ "MLX" ],
"cacheVariables": {
"MLX_CUDA_ARCHITECTURES": "86;89;90;90a;100;103;75-virtual;80-virtual;110-virtual;120-virtual;121-virtual",
"OLLAMA_RUNNER_DIR": "mlx_cuda_v13"
}
}
@@ -113,56 +40,6 @@
"configurePreset": "Default",
"configuration": "Release"
},
{
"name": "CPU",
"configurePreset": "Default",
"targets": [ "ggml-cpu" ]
},
{
"name": "CUDA",
"configurePreset": "CUDA",
"targets": [ "ggml-cuda" ]
},
{
"name": "CUDA 11",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 11"
},
{
"name": "CUDA 12",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 12"
},
{
"name": "CUDA 13",
"inherits": [ "CUDA" ],
"configurePreset": "CUDA 13"
},
{
"name": "JetPack 5",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 5"
},
{
"name": "JetPack 6",
"inherits": [ "CUDA" ],
"configurePreset": "JetPack 6"
},
{
"name": "ROCm",
"configurePreset": "ROCm",
"targets": [ "ggml-hip" ]
},
{
"name": "ROCm 6",
"inherits": [ "ROCm" ],
"configurePreset": "ROCm 6"
},
{
"name": "Vulkan",
"targets": [ "ggml-vulkan" ],
"configurePreset": "Vulkan"
},
{
"name": "MLX",
"targets": [ "mlx", "mlxc" ],
+175 -104
View File
@@ -1,28 +1,23 @@
# vim: filetype=dockerfile
ARG FLAVOR=${TARGETARCH}
ARG PARALLEL=8
ARG ROCMVERSION=6.3.3
ARG ROCMVERSION=7.2.1
ARG JETPACK5VERSION=r35.4.1
ARG JETPACK6VERSION=r36.4.0
ARG CMAKEVERSION=3.31.2
ARG NINJAVERSION=1.12.1
ARG VULKANVERSION=1.4.321.1
# Default empty stages for local MLX source overrides.
# Override with: docker build --build-context local-mlx=../mlx --build-context local-mlx-c=../mlx-c
FROM scratch AS local-mlx
FROM scratch AS local-mlx-c
FROM --platform=linux/amd64 rocm/dev-almalinux-8:${ROCMVERSION}-complete AS base-amd64
RUN dnf install -y yum-utils ccache gcc-toolset-11-gcc gcc-toolset-11-gcc-c++ gcc-toolset-11-binutils \
&& yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
ENV PATH=/opt/rh/gcc-toolset-11/root/usr/bin:$PATH
ARG VULKANVERSION
RUN wget https://sdk.lunarg.com/sdk/download/${VULKANVERSION}/linux/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz -O /tmp/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz \
&& tar xvf /tmp/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz \
&& dnf -y install ninja-build \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& /${VULKANVERSION}/vulkansdk -j 8 vulkan-headers \
&& /${VULKANVERSION}/vulkansdk -j 8 shaderc
RUN cp -r /${VULKANVERSION}/x86_64/include/* /usr/local/include/ \
&& cp -r /${VULKANVERSION}/x86_64/lib/* /usr/local/lib
ENV PATH=/${VULKANVERSION}/x86_64/bin:$PATH
FROM --platform=linux/arm64 almalinux:8 AS base-arm64
# install epel-release for ccache
@@ -33,100 +28,152 @@ ENV CC=clang CXX=clang++
FROM base-${TARGETARCH} AS base
ARG CMAKEVERSION
ARG NINJAVERSION
RUN curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1
RUN dnf install -y unzip \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux$([ "$(uname -m)" = "aarch64" ] && echo "-aarch64").zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
ENV LDFLAGS=-s
FROM base AS cpu
#
# GPU toolchain stages — provide compilers for llama-server GPU builds
#
FROM base AS cpu-deps
RUN dnf install -y gcc-toolset-11-gcc gcc-toolset-11-gcc-c++
ENV PATH=/opt/rh/gcc-toolset-11/root/usr/bin:$PATH
ARG PARALLEL
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CPU' \
&& cmake --build --parallel ${PARALLEL} --preset 'CPU' \
&& cmake --install build --component CPU --strip --parallel ${PARALLEL}
FROM base AS cuda-11
ARG CUDA11VERSION=11.8
RUN dnf install -y cuda-toolkit-${CUDA11VERSION//./-}
ENV PATH=/usr/local/cuda-11/bin:$PATH
ARG PARALLEL
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 11' \
&& cmake --build --parallel ${PARALLEL} --preset 'CUDA 11' \
&& cmake --install build --component CUDA --strip --parallel ${PARALLEL}
FROM base AS cuda-12
FROM base AS cuda-12-deps
ARG CUDA12VERSION=12.8
RUN dnf install -y cuda-toolkit-${CUDA12VERSION//./-}
ENV PATH=/usr/local/cuda-12/bin:$PATH
ARG PARALLEL
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 12' \
&& cmake --build --parallel ${PARALLEL} --preset 'CUDA 12' \
&& cmake --install build --component CUDA --strip --parallel ${PARALLEL}
FROM base AS cuda-13
FROM base AS cuda-13-deps
ARG CUDA13VERSION=13.0
RUN dnf install -y cuda-toolkit-${CUDA13VERSION//./-}
ENV PATH=/usr/local/cuda-13/bin:$PATH
ARG PARALLEL
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'CUDA 13' \
&& cmake --build --parallel ${PARALLEL} --preset 'CUDA 13' \
&& cmake --install build --component CUDA --strip --parallel ${PARALLEL}
FROM base AS rocm-7-deps
ENV PATH=/opt/rocm/llvm/bin:/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:$PATH
FROM base AS rocm-6
ENV PATH=/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin:$PATH
ARG PARALLEL
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
FROM base AS vulkan-deps
ARG VULKANVERSION
RUN ln -s /usr/bin/python3 /usr/bin/python \
&& wget https://sdk.lunarg.com/sdk/download/${VULKANVERSION}/linux/vulkansdk-linux-x86_64-${VULKANVERSION}.tar.xz -O /tmp/vulkansdk.tar.xz \
&& tar xvf /tmp/vulkansdk.tar.xz -C /tmp \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 vulkan-headers \
&& /tmp/${VULKANVERSION}/vulkansdk -j 8 shaderc \
&& cp -r /tmp/${VULKANVERSION}/x86_64/include/* /usr/local/include/ \
&& cp -r /tmp/${VULKANVERSION}/x86_64/lib/* /usr/local/lib \
&& cp -r /tmp/${VULKANVERSION}/x86_64/bin/* /usr/local/bin/ \
&& rm -rf /tmp/${VULKANVERSION} /tmp/vulkansdk.tar.xz
#
# llama-server stages — rebuild when LLAMA_CPP_VERSION, llama/server/, or llama/compat/ changes.
#
# CPU stage: llama-server + ggml-base + ggml-cpu variants → lib/ollama/
# GPU stages: GPU backend .so only → lib/ollama/<variant>/
#
FROM cpu-deps AS llama-server-cpu
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'ROCm 6' \
&& cmake --build --parallel ${PARALLEL} --preset 'ROCm 6' \
&& cmake --install build --component HIP --strip --parallel ${PARALLEL}
cmake -S llama/server --preset cpu \
&& cmake --build build/llama-server-cpu -- -l $(nproc) \
&& cmake --install build/llama-server-cpu --component llama-server --strip \
&& for lib in \
/usr/lib64/libgomp.so* \
/usr/lib64/libomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libgomp.so* \
/opt/rh/gcc-toolset-11/root/usr/lib64/libomp.so*; do \
[ -e "$lib" ] && cp -a "$lib" dist/lib/ollama/ || true; \
done
FROM cuda-12-deps AS llama-server-cuda-v12
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset cuda-v12 \
&& cmake --build build/llama-server-cuda-v12 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda-v12 --component llama-server --strip
FROM cuda-13-deps AS llama-server-cuda-v13
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset cuda-v13 \
&& cmake --build build/llama-server-cuda-v13 -- -l $(nproc) \
&& cmake --install build/llama-server-cuda-v13 --component llama-server --strip
FROM rocm-7-deps AS llama-server-rocm
ENV CC=clang CXX=clang++
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset rocm \
-DAMDGPU_TARGETS="gfx942;gfx950;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1200;gfx1201;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-" \
-DCMAKE_HIP_FLAGS="-parallel-jobs=4" \
&& cmake --build build/llama-server-rocm -- -l $(nproc) \
&& cmake --install build/llama-server-rocm --component llama-server --strip
RUN rm -f dist/lib/ollama/rocm/rocblas/library/*gfx90[06]*
FROM vulkan-deps AS llama-server-vulkan
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake -S llama/server --preset vulkan \
&& cmake --build build/llama-server-vulkan -- -l $(nproc) \
&& cmake --install build/llama-server-vulkan --component llama-server --strip
#
# JetPack stages — self-contained with their own base images
#
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK5VERSION} AS jetpack-5
ARG CMAKEVERSION
RUN apt-get update && apt-get install -y curl ccache \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
ARG PARALLEL
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 5' \
&& cmake --build --parallel ${PARALLEL} --preset 'JetPack 5' \
&& cmake --install build --component CUDA --strip --parallel ${PARALLEL}
cmake -S llama/server --preset jetpack5 \
&& cmake --build build/llama-server-jetpack5 -- -l $(nproc) \
&& cmake --install build/llama-server-jetpack5 --component llama-server --strip
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK6VERSION} AS jetpack-6
ARG CMAKEVERSION
RUN apt-get update && apt-get install -y curl ccache \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
ARG PARALLEL
ARG NINJAVERSION
RUN apt-get update && apt-get install -y curl ccache git unzip \
&& curl -fsSL https://github.com/Kitware/CMake/releases/download/v${CMAKEVERSION}/cmake-${CMAKEVERSION}-linux-$(uname -m).tar.gz | tar xz -C /usr/local --strip-components 1 \
&& curl -fsSL -o /tmp/ninja.zip https://github.com/ninja-build/ninja/releases/download/v${NINJAVERSION}/ninja-linux-aarch64.zip \
&& unzip /tmp/ninja.zip -d /usr/local/bin \
&& rm /tmp/ninja.zip
ENV CMAKE_GENERATOR=Ninja
COPY LLAMA_CPP_VERSION .
COPY llama/server llama/server
COPY llama/compat llama/compat
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'JetPack 6' \
&& cmake --build --parallel ${PARALLEL} --preset 'JetPack 6' \
&& cmake --install build --component CUDA --strip --parallel ${PARALLEL}
cmake -S llama/server --preset jetpack6 \
&& cmake --build build/llama-server-jetpack6 -- -l $(nproc) \
&& cmake --install build/llama-server-jetpack6 --component llama-server --strip
FROM base AS vulkan
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'Vulkan' \
&& cmake --build --parallel --preset 'Vulkan' \
&& cmake --install build --component Vulkan --strip --parallel 8
#
# MLX stage
#
FROM base AS mlx
ARG CUDA13VERSION=13.0
@@ -138,20 +185,30 @@ ENV PATH=/usr/local/cuda-13/bin:$PATH
ENV BLAS_INCLUDE_DIRS=/usr/include/openblas
ENV LAPACK_INCLUDE_DIRS=/usr/include/openblas
ENV CGO_LDFLAGS="-L/usr/local/cuda-13/lib64 -L/usr/local/cuda-13/targets/x86_64-linux/lib/stubs"
ARG PARALLEL
WORKDIR /go/src/github.com/ollama/ollama
COPY CMakeLists.txt CMakePresets.json .
COPY ml/backend/ggml/ggml ml/backend/ggml/ggml
COPY x/imagegen/mlx x/imagegen/mlx
COPY go.mod go.sum .
COPY MLX_VERSION .
COPY MLX_VERSION MLX_C_VERSION .
RUN curl -fsSL https://golang.org/dl/go$(awk '/^go/ { print $2 }' go.mod).linux-$(case $(uname -m) in x86_64) echo amd64 ;; aarch64) echo arm64 ;; esac).tar.gz | tar xz -C /usr/local
ENV PATH=/usr/local/go/bin:$PATH
RUN go mod download
RUN --mount=type=cache,target=/root/.ccache \
cmake --preset 'MLX CUDA 13' -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas \
&& cmake --build --parallel ${PARALLEL} --preset 'MLX CUDA 13' \
&& cmake --install build --component MLX --strip --parallel ${PARALLEL}
--mount=type=bind,from=local-mlx,target=/tmp/local-mlx \
--mount=type=bind,from=local-mlx-c,target=/tmp/local-mlx-c \
if [ -f /tmp/local-mlx/CMakeLists.txt ]; then \
export OLLAMA_MLX_SOURCE=/tmp/local-mlx; \
fi \
&& if [ -f /tmp/local-mlx-c/CMakeLists.txt ]; then \
export OLLAMA_MLX_C_SOURCE=/tmp/local-mlx-c; \
fi \
&& cmake --preset 'MLX CUDA 13' -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DLAPACK_INCLUDE_DIRS=/usr/include/openblas \
&& cmake --build --preset 'MLX CUDA 13' -- -l $(nproc) \
&& cmake --install build --component MLX --strip
#
# Go build
#
FROM base AS build
WORKDIR /go/src/github.com/ollama/ollama
@@ -160,47 +217,61 @@ RUN curl -fsSL https://golang.org/dl/go$(awk '/^go/ { print $2 }' go.mod).linux-
ENV PATH=/usr/local/go/bin:$PATH
RUN go mod download
COPY . .
# Clone mlx-c headers for CGO (version from MLX_VERSION file)
RUN git clone --depth 1 --branch "$(cat MLX_VERSION)" https://github.com/ml-explore/mlx-c.git build/_deps/mlx-c-src
ARG GOFLAGS="'-ldflags=-w -s'"
ENV CGO_ENABLED=1
ARG CGO_CFLAGS
ARG CGO_CXXFLAGS
ENV CGO_CFLAGS="${CGO_CFLAGS} -I/go/src/github.com/ollama/ollama/build/_deps/mlx-c-src"
ENV CGO_CFLAGS="${CGO_CFLAGS}"
ENV CGO_CXXFLAGS="${CGO_CXXFLAGS}"
RUN --mount=type=cache,target=/root/.cache/go-build \
go build -tags mlx -trimpath -buildmode=pie -o /bin/ollama .
go build -trimpath -buildmode=pie -o /bin/ollama .
#
# Assembly stages — combine llama-server variants + GPU runtime libs
#
FROM --platform=linux/amd64 scratch AS amd64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama /lib/ollama/
COPY --from=vulkan dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda-v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda-v13 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-vulkan dist/lib/ollama /lib/ollama/
COPY --from=mlx /go/src/github.com/ollama/ollama/dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64
# COPY --from=cuda-11 dist/lib/ollama/ /lib/ollama/
COPY --from=cuda-12 dist/lib/ollama /lib/ollama/
COPY --from=cuda-13 dist/lib/ollama/ /lib/ollama/
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda-v12 dist/lib/ollama /lib/ollama/
COPY --from=llama-server-cuda-v13 dist/lib/ollama /lib/ollama/
COPY --from=jetpack-5 dist/lib/ollama/ /lib/ollama/
COPY --from=jetpack-6 dist/lib/ollama/ /lib/ollama/
FROM scratch AS rocm
COPY --from=rocm-6 dist/lib/ollama /lib/ollama
COPY --from=llama-server-cpu dist/lib/ollama /lib/ollama
COPY --from=llama-server-rocm dist/lib/ollama /lib/ollama
FROM ${FLAVOR} AS archive
ARG VULKANVERSION
COPY --from=cpu dist/lib/ollama /lib/ollama
FROM --platform=linux/amd64 scratch AS amd64-archive
COPY --from=amd64 /lib/ollama /lib/ollama/
COPY --from=llama-server-rocm dist/lib/ollama /lib/ollama/
FROM --platform=linux/arm64 scratch AS arm64-archive
COPY --from=arm64 /lib/ollama /lib/ollama/
FROM ${TARGETARCH}-archive AS archive
COPY --from=build /bin/ollama /bin/ollama
FROM ${FLAVOR} AS image-archive
COPY --from=build /bin/ollama /bin/ollama
FROM ubuntu:24.04
RUN apt-get update \
ARG APT_MIRROR=http://archive.ubuntu.com/ubuntu
RUN sed -i "s|http://archive.ubuntu.com/ubuntu|$APT_MIRROR|g" /etc/apt/sources.list.d/ubuntu.sources \
&& apt-get update \
&& apt-get install -y ca-certificates libvulkan1 libopenblas0 \
&& sed -i "s|$APT_MIRROR|http://archive.ubuntu.com/ubuntu|g" /etc/apt/sources.list.d/ubuntu.sources \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY --from=archive /bin /usr/bin
COPY --from=image-archive /bin /usr/bin
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
COPY --from=archive /lib/ollama /usr/lib/ollama
COPY --from=image-archive /lib/ollama /usr/lib/ollama
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_VISIBLE_DEVICES=all
+1
View File
@@ -0,0 +1 @@
b9048
+1
View File
@@ -0,0 +1 @@
fba4470b89073180056c9ea46c443051375f7399
+1 -1
View File
@@ -1 +1 @@
v0.5.0
e8ebdebeeb655feaa85a51f6b24ece5b6d5518d1
-76
View File
@@ -1,76 +0,0 @@
UPSTREAM=https://github.com/ggml-org/llama.cpp.git
WORKDIR=llama/vendor
FETCH_HEAD=ec98e2002
.PHONY: help
help:
@echo "Available targets:"
@echo " sync Sync with upstream repositories"
@echo " checkout Checkout upstream repository"
@echo " apply-patches Apply patches to local repository"
@echo " format-patches Format patches from local repository"
@echo " clean Clean local repository"
@echo
@echo "Example:"
@echo " make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches sync"
.PHONY: sync
sync: llama/build-info.cpp ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal
llama/build-info.cpp: llama/build-info.cpp.in llama/llama.cpp
sed -e 's|@FETCH_HEAD@|$(FETCH_HEAD)|' <$< >$@
ml/backend/ggml/ggml/src/ggml-metal/ggml-metal-embed.metal: ml/backend/ggml/ggml
go generate ./$(@D)
.PHONY: llama/llama.cpp
llama/llama.cpp: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /) $@
.PHONY: ml/backend/ggml/ggml
ml/backend/ggml/ggml: llama/vendor
rsync -arvzc --delete -f "include LICENSE" -f "merge $@/.rsync-filter" $(addprefix $<,/LICENSE /ggml/) $@
PATCHES=$(wildcard llama/patches/*.patch)
PATCHED=$(join $(dir $(PATCHES)), $(addsuffix ed, $(addprefix ., $(notdir $(PATCHES)))))
.PHONY: apply-patches
.NOTPARALLEL:
apply-patches: $(PATCHED)
llama/patches/.%.patched: llama/patches/%.patch
@if git -c user.name=nobody -c 'user.email=<>' -C $(WORKDIR) am -3 $(realpath $<); then \
touch $@; \
else \
echo "Patch failed. Resolve any conflicts then continue."; \
echo "1. Run 'git -C $(WORKDIR) am --continue'"; \
echo "2. Run 'make -f $(lastword $(MAKEFILE_LIST)) format-patches'"; \
echo "3. Run 'make -f $(lastword $(MAKEFILE_LIST)) clean apply-patches'"; \
exit 1; \
fi
.PHONY: checkout
checkout: $(WORKDIR)
git -C $(WORKDIR) fetch
git -C $(WORKDIR) checkout -f $(FETCH_HEAD)
$(WORKDIR):
git clone $(UPSTREAM) $(WORKDIR)
.PHONY: format-patches
format-patches: llama/patches
git -C $(WORKDIR) format-patch \
--no-signature \
--no-numbered \
--zero-commit \
-o $(realpath $<) \
$(FETCH_HEAD)
.PHONY: clean
clean: checkout
@git -C $(WORKDIR) am --abort || true
$(RM) llama/patches/.*.patched
.PHONY: print-base
print-base:
@echo $(FETCH_HEAD)
+2 -2
View File
@@ -55,7 +55,7 @@ The official [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) `olla
ollama
```
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as `claude`, `codex`, `openclaw` and more.
You'll be prompted to run a model or connect Ollama to your existing agents or applications such as `Claude Code`, `OpenClaw`, `OpenCode` , `Codex`, `Copilot`, and more.
### Coding
@@ -65,7 +65,7 @@ To launch a specific integration:
ollama launch claude
```
Supported integrations include [Claude Code](https://docs.ollama.com/integrations/claude-code), [Codex](https://docs.ollama.com/integrations/codex), [Droid](https://docs.ollama.com/integrations/droid), and [OpenCode](https://docs.ollama.com/integrations/opencode).
Supported integrations include [Claude Code](https://docs.ollama.com/integrations/claude-code), [Codex](https://docs.ollama.com/integrations/codex), [Copilot CLI](https://docs.ollama.com/integrations/copilot-cli), [Droid](https://docs.ollama.com/integrations/droid), and [OpenCode](https://docs.ollama.com/integrations/opencode).
### AI assistant
+183 -204
View File
@@ -68,7 +68,7 @@ type MessagesRequest struct {
Model string `json:"model"`
MaxTokens int `json:"max_tokens"`
Messages []MessageParam `json:"messages"`
System any `json:"system,omitempty"` // string or []ContentBlock
System any `json:"system,omitempty"` // string or []map[string]any (JSON-decoded ContentBlock)
Stream bool `json:"stream,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
@@ -82,8 +82,27 @@ type MessagesRequest struct {
// MessageParam represents a message in the request
type MessageParam struct {
Role string `json:"role"` // "user" or "assistant"
Content any `json:"content"` // string or []ContentBlock
Role string `json:"role"` // "user" or "assistant"
Content []ContentBlock `json:"content"` // always []ContentBlock; plain strings are normalized on unmarshal
}
func (m *MessageParam) UnmarshalJSON(data []byte) error {
var raw struct {
Role string `json:"role"`
Content json.RawMessage `json:"content"`
}
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
m.Role = raw.Role
var s string
if err := json.Unmarshal(raw.Content, &s); err == nil {
m.Content = []ContentBlock{{Type: "text", Text: &s}}
return nil
}
return json.Unmarshal(raw.Content, &m.Content)
}
// ContentBlock represents a content block in a message.
@@ -102,9 +121,9 @@ type ContentBlock struct {
Source *ImageSource `json:"source,omitempty"`
// For tool_use and server_tool_use blocks
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input any `json:"input,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Input api.ToolCallFunctionArguments `json:"input,omitzero"`
// For tool_result and web_search_tool_result blocks
ToolUseID string `json:"tool_use_id,omitempty"`
@@ -377,178 +396,145 @@ func convertMessage(msg MessageParam) ([]api.Message, error) {
var messages []api.Message
role := strings.ToLower(msg.Role)
switch content := msg.Content.(type) {
case string:
messages = append(messages, api.Message{Role: role, Content: content})
var textContent strings.Builder
var images []api.ImageData
var toolCalls []api.ToolCall
var thinking string
var toolResults []api.Message
textBlocks := 0
imageBlocks := 0
toolUseBlocks := 0
toolResultBlocks := 0
serverToolUseBlocks := 0
webSearchToolResultBlocks := 0
thinkingBlocks := 0
unknownBlocks := 0
case []any:
var textContent strings.Builder
var images []api.ImageData
var toolCalls []api.ToolCall
var thinking string
var toolResults []api.Message
textBlocks := 0
imageBlocks := 0
toolUseBlocks := 0
toolResultBlocks := 0
serverToolUseBlocks := 0
webSearchToolResultBlocks := 0
thinkingBlocks := 0
unknownBlocks := 0
for _, block := range content {
blockMap, ok := block.(map[string]any)
if !ok {
logutil.Trace("anthropic: invalid content block format", "role", role)
return nil, errors.New("invalid content block format")
for _, block := range msg.Content {
switch block.Type {
case "text":
textBlocks++
if block.Text != nil {
textContent.WriteString(*block.Text)
}
blockType, _ := blockMap["type"].(string)
case "image":
imageBlocks++
if block.Source == nil {
logutil.Trace("anthropic: invalid image source", "role", role)
return nil, errors.New("invalid image source")
}
switch blockType {
case "text":
textBlocks++
if text, ok := blockMap["text"].(string); ok {
textContent.WriteString(text)
if block.Source.Type == "base64" {
decoded, err := base64.StdEncoding.DecodeString(block.Source.Data)
if err != nil {
logutil.Trace("anthropic: invalid base64 image data", "role", role, "error", err)
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
images = append(images, decoded)
} else {
logutil.Trace("anthropic: unsupported image source type", "role", role, "source_type", block.Source.Type)
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", block.Source.Type)
}
case "image":
imageBlocks++
source, ok := blockMap["source"].(map[string]any)
if !ok {
logutil.Trace("anthropic: invalid image source", "role", role)
return nil, errors.New("invalid image source")
}
case "tool_use":
toolUseBlocks++
if block.ID == "" {
logutil.Trace("anthropic: tool_use block missing id", "role", role)
return nil, errors.New("tool_use block missing required 'id' field")
}
if block.Name == "" {
logutil.Trace("anthropic: tool_use block missing name", "role", role)
return nil, errors.New("tool_use block missing required 'name' field")
}
toolCalls = append(toolCalls, api.ToolCall{
ID: block.ID,
Function: api.ToolCallFunction{
Name: block.Name,
Arguments: block.Input,
},
})
sourceType, _ := source["type"].(string)
if sourceType == "base64" {
data, _ := source["data"].(string)
decoded, err := base64.StdEncoding.DecodeString(data)
if err != nil {
logutil.Trace("anthropic: invalid base64 image data", "role", role, "error", err)
return nil, fmt.Errorf("invalid base64 image data: %w", err)
}
images = append(images, decoded)
} else {
logutil.Trace("anthropic: unsupported image source type", "role", role, "source_type", sourceType)
return nil, fmt.Errorf("invalid image source type: %s. Only base64 images are supported.", sourceType)
}
// URL images would need to be fetched - skip for now
case "tool_result":
toolResultBlocks++
var resultContent string
case "tool_use":
toolUseBlocks++
id, ok := blockMap["id"].(string)
if !ok {
logutil.Trace("anthropic: tool_use block missing id", "role", role)
return nil, errors.New("tool_use block missing required 'id' field")
}
name, ok := blockMap["name"].(string)
if !ok {
logutil.Trace("anthropic: tool_use block missing name", "role", role)
return nil, errors.New("tool_use block missing required 'name' field")
}
tc := api.ToolCall{
ID: id,
Function: api.ToolCallFunction{
Name: name,
},
}
if input, ok := blockMap["input"].(map[string]any); ok {
tc.Function.Arguments = mapToArgs(input)
}
toolCalls = append(toolCalls, tc)
case "tool_result":
toolResultBlocks++
toolUseID, _ := blockMap["tool_use_id"].(string)
var resultContent string
switch c := blockMap["content"].(type) {
case string:
resultContent = c
case []any:
for _, cb := range c {
if cbMap, ok := cb.(map[string]any); ok {
if cbMap["type"] == "text" {
if text, ok := cbMap["text"].(string); ok {
resultContent += text
}
switch c := block.Content.(type) {
case string:
resultContent = c
case []any:
for _, cb := range c {
if cbMap, ok := cb.(map[string]any); ok {
if cbMap["type"] == "text" {
if text, ok := cbMap["text"].(string); ok {
resultContent += text
}
}
}
}
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: resultContent,
ToolCallID: toolUseID,
})
case "thinking":
thinkingBlocks++
if t, ok := blockMap["thinking"].(string); ok {
thinking = t
}
case "server_tool_use":
serverToolUseBlocks++
id, _ := blockMap["id"].(string)
name, _ := blockMap["name"].(string)
tc := api.ToolCall{
ID: id,
Function: api.ToolCallFunction{
Name: name,
},
}
if input, ok := blockMap["input"].(map[string]any); ok {
tc.Function.Arguments = mapToArgs(input)
}
toolCalls = append(toolCalls, tc)
case "web_search_tool_result":
webSearchToolResultBlocks++
toolUseID, _ := blockMap["tool_use_id"].(string)
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: formatWebSearchToolResultContent(blockMap["content"]),
ToolCallID: toolUseID,
})
default:
unknownBlocks++
}
}
if textContent.Len() > 0 || len(images) > 0 || len(toolCalls) > 0 || thinking != "" {
m := api.Message{
Role: role,
Content: textContent.String(),
Images: images,
ToolCalls: toolCalls,
Thinking: thinking,
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: resultContent,
ToolCallID: block.ToolUseID,
})
case "thinking":
thinkingBlocks++
if block.Thinking != nil {
thinking = *block.Thinking
}
messages = append(messages, m)
case "server_tool_use":
serverToolUseBlocks++
toolCalls = append(toolCalls, api.ToolCall{
ID: block.ID,
Function: api.ToolCallFunction{
Name: block.Name,
Arguments: block.Input,
},
})
case "web_search_tool_result":
webSearchToolResultBlocks++
toolResults = append(toolResults, api.Message{
Role: "tool",
Content: formatWebSearchToolResultContent(block.Content),
ToolCallID: block.ToolUseID,
})
default:
unknownBlocks++
}
// Add tool results as separate messages
messages = append(messages, toolResults...)
logutil.Trace("anthropic: converted block message",
"role", role,
"blocks", len(content),
"text", textBlocks,
"image", imageBlocks,
"tool_use", toolUseBlocks,
"tool_result", toolResultBlocks,
"server_tool_use", serverToolUseBlocks,
"web_search_result", webSearchToolResultBlocks,
"thinking", thinkingBlocks,
"unknown", unknownBlocks,
"messages", TraceAPIMessages(messages),
)
default:
return nil, fmt.Errorf("invalid message content type: %T", content)
}
if textContent.Len() > 0 || len(images) > 0 || len(toolCalls) > 0 || thinking != "" {
m := api.Message{
Role: role,
Content: textContent.String(),
Images: images,
ToolCalls: toolCalls,
Thinking: thinking,
}
messages = append(messages, m)
}
// Add tool results as separate messages
messages = append(messages, toolResults...)
logutil.Trace("anthropic: converted block message",
"role", role,
"blocks", len(msg.Content),
"text", textBlocks,
"image", imageBlocks,
"tool_use", toolUseBlocks,
"tool_result", toolResultBlocks,
"server_tool_use", serverToolUseBlocks,
"web_search_result", webSearchToolResultBlocks,
"thinking", thinkingBlocks,
"unknown", unknownBlocks,
"messages", TraceAPIMessages(messages),
)
return messages, nil
}
@@ -852,6 +838,19 @@ func (c *StreamConverter) Process(r api.ChatResponse) []StreamEvent {
continue
}
// Close thinking block if still open (thinking → tool_use without text in between)
if c.thinkingStarted && !c.thinkingDone {
c.thinkingDone = true
events = append(events, StreamEvent{
Event: "content_block_stop",
Data: ContentBlockStopEvent{
Type: "content_block_stop",
Index: c.contentIndex,
},
})
c.contentIndex++
}
if c.textStarted {
events = append(events, StreamEvent{
Event: "content_block_stop",
@@ -869,7 +868,6 @@ func (c *StreamConverter) Process(r api.ChatResponse) []StreamEvent {
slog.Error("failed to marshal tool arguments", "error", err, "tool_id", tc.ID)
continue
}
events = append(events, StreamEvent{
Event: "content_block_start",
Data: ContentBlockStartEvent{
@@ -879,7 +877,7 @@ func (c *StreamConverter) Process(r api.ChatResponse) []StreamEvent {
Type: "tool_use",
ID: tc.ID,
Name: tc.Function.Name,
Input: map[string]any{},
Input: api.NewToolCallFunctionArguments(),
},
},
})
@@ -976,15 +974,6 @@ func ptr(s string) *string {
return &s
}
// mapToArgs converts a map to ToolCallFunctionArguments
func mapToArgs(m map[string]any) api.ToolCallFunctionArguments {
args := api.NewToolCallFunctionArguments()
for k, v := range m {
args.Set(k, v)
}
return args
}
// CountTokensRequest represents an Anthropic count_tokens request
type CountTokensRequest struct {
Model string `json:"model"`
@@ -1017,17 +1006,13 @@ func estimateTokens(req CountTokensRequest) int {
var totalLen int
// Count system prompt
if req.System != nil {
totalLen += countAnyContent(req.System)
}
totalLen += countAnyContent(req.System)
// Count messages
for _, msg := range req.Messages {
// Count role (always present)
totalLen += len(msg.Role)
// Count content
contentLen := countAnyContent(msg.Content)
totalLen += contentLen
totalLen += countAnyContent(msg.Content)
}
for _, tool := range req.Tools {
@@ -1050,12 +1035,25 @@ func countAnyContent(content any) int {
switch c := content.(type) {
case string:
return len(c)
case []any:
case []ContentBlock:
total := 0
for _, block := range c {
total += countContentBlock(block)
}
return total
case []any:
total := 0
for _, item := range c {
data, err := json.Marshal(item)
if err != nil {
continue
}
var block ContentBlock
if err := json.Unmarshal(data, &block); err == nil {
total += countContentBlock(block)
}
}
return total
default:
if data, err := json.Marshal(content); err == nil {
return len(data)
@@ -1064,38 +1062,19 @@ func countAnyContent(content any) int {
}
}
func countContentBlock(block any) int {
blockMap, ok := block.(map[string]any)
if !ok {
if s, ok := block.(string); ok {
return len(s)
}
return 0
}
func countContentBlock(block ContentBlock) int {
total := 0
blockType, _ := blockMap["type"].(string)
if text, ok := blockMap["text"].(string); ok {
total += len(text)
if block.Text != nil {
total += len(*block.Text)
}
if thinking, ok := blockMap["thinking"].(string); ok {
total += len(thinking)
if block.Thinking != nil {
total += len(*block.Thinking)
}
if blockType == "tool_use" {
if data, err := json.Marshal(blockMap); err == nil {
if block.Type == "tool_use" || block.Type == "tool_result" {
if data, err := json.Marshal(block); err == nil {
total += len(data)
}
}
if blockType == "tool_result" {
if data, err := json.Marshal(blockMap); err == nil {
total += len(data)
}
}
return total
}
+299 -82
View File
@@ -15,11 +15,16 @@ const (
testImage = `iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=`
)
// testArgs creates ToolCallFunctionArguments from a map (convenience function for tests)
func testArgs(m map[string]any) api.ToolCallFunctionArguments {
// textContent is a convenience for constructing []ContentBlock with a single text block in tests.
func textContent(s string) []ContentBlock {
return []ContentBlock{{Type: "text", Text: &s}}
}
// makeArgs creates ToolCallFunctionArguments from key-value pairs (convenience function for tests)
func makeArgs(kvs ...any) api.ToolCallFunctionArguments {
args := api.NewToolCallFunctionArguments()
for k, v := range m {
args.Set(k, v)
for i := 0; i < len(kvs)-1; i += 2 {
args.Set(kvs[i].(string), kvs[i+1])
}
return args
}
@@ -29,7 +34,7 @@ func TestFromMessagesRequest_Basic(t *testing.T) {
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
},
}
@@ -61,7 +66,7 @@ func TestFromMessagesRequest_WithSystemPrompt(t *testing.T) {
MaxTokens: 1024,
System: "You are a helpful assistant.",
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
},
}
@@ -88,7 +93,7 @@ func TestFromMessagesRequest_WithSystemPromptArray(t *testing.T) {
map[string]any{"type": "text", "text": " Be concise."},
},
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
},
}
@@ -113,7 +118,7 @@ func TestFromMessagesRequest_WithOptions(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 2048,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Temperature: &temp,
TopP: &topP,
TopK: &topK,
@@ -148,14 +153,14 @@ func TestFromMessagesRequest_WithImage(t *testing.T) {
Messages: []MessageParam{
{
Role: "user",
Content: []any{
map[string]any{"type": "text", "text": "What's in this image?"},
map[string]any{
"type": "image",
"source": map[string]any{
"type": "base64",
"media_type": "image/png",
"data": testImage,
Content: []ContentBlock{
{Type: "text", Text: ptr("What's in this image?")},
{
Type: "image",
Source: &ImageSource{
Type: "base64",
MediaType: "image/png",
Data: testImage,
},
},
},
@@ -190,15 +195,15 @@ func TestFromMessagesRequest_WithToolUse(t *testing.T) {
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{Role: "user", Content: "What's the weather in Paris?"},
{Role: "user", Content: textContent("What's the weather in Paris?")},
{
Role: "assistant",
Content: []any{
map[string]any{
"type": "tool_use",
"id": "call_123",
"name": "get_weather",
"input": map[string]any{"location": "Paris"},
Content: []ContentBlock{
{
Type: "tool_use",
ID: "call_123",
Name: "get_weather",
Input: makeArgs("location", "Paris"),
},
},
},
@@ -234,11 +239,11 @@ func TestFromMessagesRequest_WithToolResult(t *testing.T) {
Messages: []MessageParam{
{
Role: "user",
Content: []any{
map[string]any{
"type": "tool_result",
"tool_use_id": "call_123",
"content": "The weather in Paris is sunny, 22°C",
Content: []ContentBlock{
{
Type: "tool_result",
ToolUseID: "call_123",
Content: "The weather in Paris is sunny, 22°C",
},
},
},
@@ -270,7 +275,7 @@ func TestFromMessagesRequest_WithTools(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Tools: []Tool{
{
Name: "get_weather",
@@ -305,7 +310,7 @@ func TestFromMessagesRequest_DropsCustomWebSearchWhenBuiltinPresent(t *testing.T
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Tools: []Tool{
{
Type: "web_search_20250305",
@@ -346,7 +351,7 @@ func TestFromMessagesRequest_KeepsCustomWebSearchWhenBuiltinAbsent(t *testing.T)
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Tools: []Tool{
{
Type: "custom",
@@ -377,7 +382,7 @@ func TestFromMessagesRequest_WithThinking(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Thinking: &ThinkingConfig{Type: "enabled", BudgetTokens: 1000},
}
@@ -399,13 +404,13 @@ func TestFromMessagesRequest_ThinkingOnlyBlock(t *testing.T) {
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
{
Role: "assistant",
Content: []any{
map[string]any{
"type": "thinking",
"thinking": "Let me think about this...",
Content: []ContentBlock{
{
Type: "thinking",
Thinking: ptr("Let me think about this..."),
},
},
},
@@ -434,10 +439,10 @@ func TestFromMessagesRequest_ToolUseMissingID(t *testing.T) {
Messages: []MessageParam{
{
Role: "assistant",
Content: []any{
map[string]any{
"type": "tool_use",
"name": "get_weather",
Content: []ContentBlock{
{
Type: "tool_use",
Name: "get_weather",
},
},
},
@@ -460,10 +465,10 @@ func TestFromMessagesRequest_ToolUseMissingName(t *testing.T) {
Messages: []MessageParam{
{
Role: "assistant",
Content: []any{
map[string]any{
"type": "tool_use",
"id": "call_123",
Content: []ContentBlock{
{
Type: "tool_use",
ID: "call_123",
},
},
},
@@ -483,7 +488,7 @@ func TestFromMessagesRequest_InvalidToolSchema(t *testing.T) {
req := MessagesRequest{
Model: "test-model",
MaxTokens: 1024,
Messages: []MessageParam{{Role: "user", Content: "Hello"}},
Messages: []MessageParam{{Role: "user", Content: textContent("Hello")}},
Tools: []Tool{
{
Name: "bad_tool",
@@ -548,7 +553,7 @@ func TestToMessagesResponse_WithToolCalls(t *testing.T) {
ID: "call_123",
Function: api.ToolCallFunction{
Name: "get_weather",
Arguments: testArgs(map[string]any{"location": "Paris"}),
Arguments: makeArgs("location", "Paris"),
},
},
},
@@ -760,7 +765,7 @@ func TestStreamConverter_WithToolCalls(t *testing.T) {
ID: "call_123",
Function: api.ToolCallFunction{
Name: "get_weather",
Arguments: testArgs(map[string]any{"location": "Paris"}),
Arguments: makeArgs("location", "Paris"),
},
},
},
@@ -799,6 +804,107 @@ func TestStreamConverter_WithToolCalls(t *testing.T) {
}
}
// TestStreamConverter_ThinkingDirectlyFollowedByToolCall verifies that when a
// model emits a thinking block followed directly by a tool_use block (with no
// text block in between), the streaming converter correctly closes the thinking
// block and increments the content index before opening the tool_use block.
// Previously, the converter reused contentIndex=0 for the tool_use block,
// which caused "Content block not found" errors in clients. See #14816.
func TestStreamConverter_ThinkingDirectlyFollowedByToolCall(t *testing.T) {
conv := NewStreamConverter("msg_123", "test-model", 0)
// First chunk: thinking content (no text)
resp1 := api.ChatResponse{
Model: "test-model",
Message: api.Message{
Role: "assistant",
Thinking: "I should call the tool.",
},
}
events1 := conv.Process(resp1)
// Should have: message_start, content_block_start(thinking), content_block_delta(thinking)
if len(events1) < 3 {
t.Fatalf("expected at least 3 events for thinking chunk, got %d", len(events1))
}
if events1[0].Event != "message_start" {
t.Errorf("expected first event 'message_start', got %q", events1[0].Event)
}
thinkingStart, ok := events1[1].Data.(ContentBlockStartEvent)
if !ok || thinkingStart.ContentBlock.Type != "thinking" {
t.Errorf("expected content_block_start(thinking) as second event, got %+v", events1[1])
}
if thinkingStart.Index != 0 {
t.Errorf("expected thinking block at index 0, got %d", thinkingStart.Index)
}
// Second chunk: tool call (no text between thinking and tool)
resp2 := api.ChatResponse{
Model: "test-model",
Message: api.Message{
Role: "assistant",
ToolCalls: []api.ToolCall{
{
ID: "call_abc",
Function: api.ToolCallFunction{
Name: "ask_user",
Arguments: makeArgs("question", "cats or dogs?"),
},
},
},
},
Done: true,
DoneReason: "stop",
Metrics: api.Metrics{PromptEvalCount: 10, EvalCount: 5},
}
events2 := conv.Process(resp2)
// Expect: content_block_stop(index=0), content_block_start(tool_use, index=1),
// content_block_delta(input_json_delta, index=1), content_block_stop(index=1),
// message_delta, message_stop
var thinkingStop, toolStart, toolDelta, toolStop *StreamEvent
for i := range events2 {
e := &events2[i]
switch e.Event {
case "content_block_stop":
if stop, ok := e.Data.(ContentBlockStopEvent); ok {
if stop.Index == 0 && thinkingStop == nil {
thinkingStop = e
} else if stop.Index == 1 {
toolStop = e
}
}
case "content_block_start":
if start, ok := e.Data.(ContentBlockStartEvent); ok && start.ContentBlock.Type == "tool_use" {
toolStart = e
}
case "content_block_delta":
if delta, ok := e.Data.(ContentBlockDeltaEvent); ok && delta.Delta.Type == "input_json_delta" {
toolDelta = e
}
}
}
if thinkingStop == nil {
t.Error("expected content_block_stop for thinking block (index 0)")
}
if toolStart == nil {
t.Fatal("expected content_block_start for tool_use block")
}
if start, ok := toolStart.Data.(ContentBlockStartEvent); !ok || start.Index != 1 {
t.Errorf("expected tool_use block at index 1, got %+v", toolStart.Data)
}
if toolDelta == nil {
t.Fatal("expected input_json_delta event for tool call")
}
if delta, ok := toolDelta.Data.(ContentBlockDeltaEvent); !ok || delta.Index != 1 {
t.Errorf("expected tool delta at index 1, got %+v", toolDelta.Data)
}
if toolStop == nil {
t.Error("expected content_block_stop for tool_use block (index 1)")
}
}
func TestStreamConverter_ToolCallWithUnmarshalableArgs(t *testing.T) {
// Test that unmarshalable arguments (like channels) are handled gracefully
// and don't cause a panic or corrupt stream
@@ -864,7 +970,7 @@ func TestStreamConverter_MultipleToolCallsWithMixedValidity(t *testing.T) {
ID: "call_good",
Function: api.ToolCallFunction{
Name: "good_function",
Arguments: testArgs(map[string]any{"location": "Paris"}),
Arguments: makeArgs("location", "Paris"),
},
},
{
@@ -966,6 +1072,57 @@ func TestContentBlockJSON_EmptyFieldsPresent(t *testing.T) {
}
}
func TestContentBlockJSON_NonToolBlocksDoNotIncludeInput(t *testing.T) {
tests := []struct {
name string
block ContentBlock
}{
{
name: "text block",
block: ContentBlock{
Type: "text",
Text: ptr("hello"),
},
},
{
name: "thinking block",
block: ContentBlock{
Type: "thinking",
Thinking: ptr("let me think"),
},
},
{
name: "image block",
block: ContentBlock{
Type: "image",
Source: &ImageSource{
Type: "base64",
MediaType: "image/png",
Data: testImage,
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
data, err := json.Marshal(tt.block)
if err != nil {
t.Fatalf("failed to marshal: %v", err)
}
var result map[string]any
if err := json.Unmarshal(data, &result); err != nil {
t.Fatalf("failed to unmarshal: %v", err)
}
if _, ok := result["input"]; ok {
t.Fatalf("unexpected input field in non-tool block JSON: %s", string(data))
}
})
}
}
func TestStreamConverter_ContentBlockStartIncludesEmptyFields(t *testing.T) {
t.Run("text block start includes empty text", func(t *testing.T) {
conv := NewStreamConverter("msg_123", "test-model", 0)
@@ -986,7 +1143,9 @@ func TestStreamConverter_ContentBlockStartIncludesEmptyFields(t *testing.T) {
// Marshal and verify the text field is present
data, _ := json.Marshal(start)
var result map[string]any
json.Unmarshal(data, &result)
if err := json.Unmarshal(data, &result); err != nil {
t.Fatalf("failed to unmarshal content_block_start JSON: %v", err)
}
cb := result["content_block"].(map[string]any)
if _, ok := cb["text"]; !ok {
t.Error("content_block_start for text should include 'text' field")
@@ -1033,13 +1192,71 @@ func TestStreamConverter_ContentBlockStartIncludesEmptyFields(t *testing.T) {
t.Error("expected thinking content_block_start event")
}
})
t.Run("tool_use block start includes empty input object", func(t *testing.T) {
conv := NewStreamConverter("msg_123", "test-model", 0)
resp := api.ChatResponse{
Model: "test-model",
Message: api.Message{
Role: "assistant",
ToolCalls: []api.ToolCall{
{
ID: "call_123",
Function: api.ToolCallFunction{
Name: "get_weather",
Arguments: makeArgs("location", "Paris"),
},
},
},
},
}
events := conv.Process(resp)
var foundToolStart bool
for _, e := range events {
if e.Event == "content_block_start" {
if start, ok := e.Data.(ContentBlockStartEvent); ok {
if start.ContentBlock.Type == "tool_use" {
foundToolStart = true
if start.ContentBlock.Input.Len() != 0 {
t.Errorf("expected empty input object, got len=%d", start.ContentBlock.Input.Len())
}
data, _ := json.Marshal(start)
var result map[string]any
json.Unmarshal(data, &result)
cb := result["content_block"].(map[string]any)
input, ok := cb["input"]
if !ok {
t.Error("content_block_start for tool_use should include 'input' field")
continue
}
inputMap, ok := input.(map[string]any)
if !ok {
t.Errorf("input field should be an object, got %T", input)
continue
}
if len(inputMap) != 0 {
t.Errorf("expected empty input object in content_block_start, got %v", inputMap)
}
}
}
}
}
if !foundToolStart {
t.Error("expected tool_use content_block_start event")
}
})
}
func TestEstimateTokens_SimpleMessage(t *testing.T) {
req := CountTokensRequest{
Model: "test-model",
Messages: []MessageParam{
{Role: "user", Content: "Hello, world!"},
{Role: "user", Content: textContent("Hello, world!")},
},
}
@@ -1060,7 +1277,7 @@ func TestEstimateTokens_WithSystemPrompt(t *testing.T) {
Model: "test-model",
System: "You are a helpful assistant.",
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
},
}
@@ -1076,7 +1293,7 @@ func TestEstimateTokens_WithTools(t *testing.T) {
req := CountTokensRequest{
Model: "test-model",
Messages: []MessageParam{
{Role: "user", Content: "What's the weather?"},
{Role: "user", Content: textContent("What's the weather?")},
},
Tools: []Tool{
{
@@ -1099,17 +1316,17 @@ func TestEstimateTokens_WithThinking(t *testing.T) {
req := CountTokensRequest{
Model: "test-model",
Messages: []MessageParam{
{Role: "user", Content: "Hello"},
{Role: "user", Content: textContent("Hello")},
{
Role: "assistant",
Content: []any{
map[string]any{
"type": "thinking",
"thinking": "Let me think about this carefully...",
Content: []ContentBlock{
{
Type: "thinking",
Thinking: ptr("Let me think about this carefully..."),
},
map[string]any{
"type": "text",
"text": "Here is my response.",
{
Type: "text",
Text: ptr("Here is my response."),
},
},
},
@@ -1207,12 +1424,12 @@ func TestConvertTool_RegularTool(t *testing.T) {
func TestConvertMessage_ServerToolUse(t *testing.T) {
msg := MessageParam{
Role: "assistant",
Content: []any{
map[string]any{
"type": "server_tool_use",
"id": "srvtoolu_123",
"name": "web_search",
"input": map[string]any{"query": "test query"},
Content: []ContentBlock{
{
Type: "server_tool_use",
ID: "srvtoolu_123",
Name: "web_search",
Input: makeArgs("query", "test query"),
},
},
}
@@ -1243,11 +1460,11 @@ func TestConvertMessage_ServerToolUse(t *testing.T) {
func TestConvertMessage_WebSearchToolResult(t *testing.T) {
msg := MessageParam{
Role: "user",
Content: []any{
map[string]any{
"type": "web_search_tool_result",
"tool_use_id": "srvtoolu_123",
"content": []any{
Content: []ContentBlock{
{
Type: "web_search_tool_result",
ToolUseID: "srvtoolu_123",
Content: []any{
map[string]any{
"type": "web_search_result",
"title": "Test Result",
@@ -1284,11 +1501,11 @@ func TestConvertMessage_WebSearchToolResult(t *testing.T) {
func TestConvertMessage_WebSearchToolResultEmptyStillCreatesToolMessage(t *testing.T) {
msg := MessageParam{
Role: "user",
Content: []any{
map[string]any{
"type": "web_search_tool_result",
"tool_use_id": "srvtoolu_empty",
"content": []any{},
Content: []ContentBlock{
{
Type: "web_search_tool_result",
ToolUseID: "srvtoolu_empty",
Content: []any{},
},
},
}
@@ -1315,11 +1532,11 @@ func TestConvertMessage_WebSearchToolResultEmptyStillCreatesToolMessage(t *testi
func TestConvertMessage_WebSearchToolResultErrorStillCreatesToolMessage(t *testing.T) {
msg := MessageParam{
Role: "user",
Content: []any{
map[string]any{
"type": "web_search_tool_result",
"tool_use_id": "srvtoolu_error",
"content": map[string]any{
Content: []ContentBlock{
{
Type: "web_search_tool_result",
ToolUseID: "srvtoolu_error",
Content: map[string]any{
"type": "web_search_tool_result_error",
"error_code": "max_uses_exceeded",
},
+10 -22
View File
@@ -368,6 +368,16 @@ func (c *Client) List(ctx context.Context) (*ListResponse, error) {
return &lr, nil
}
// ModelRecommendationsExperimental lists model recommendations from the local
// server's experimental recommendations endpoint.
func (c *Client) ModelRecommendationsExperimental(ctx context.Context) (*ModelRecommendationsResponse, error) {
var resp ModelRecommendationsResponse
if err := c.do(ctx, http.MethodGet, "/api/experimental/model-recommendations", nil, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// ListRunning lists running models.
func (c *Client) ListRunning(ctx context.Context) (*ProcessResponse, error) {
var lr ProcessResponse
@@ -476,25 +486,3 @@ func (c *Client) Whoami(ctx context.Context) (*UserResponse, error) {
}
return &resp, nil
}
// AliasRequest is the request body for creating or updating a model alias.
type AliasRequest struct {
Alias string `json:"alias"`
Target string `json:"target"`
PrefixMatching bool `json:"prefix_matching,omitempty"`
}
// SetAliasExperimental creates or updates a model alias via the experimental aliases API.
func (c *Client) SetAliasExperimental(ctx context.Context, req *AliasRequest) error {
return c.do(ctx, http.MethodPost, "/api/experimental/aliases", req, nil)
}
// AliasDeleteRequest is the request body for deleting a model alias.
type AliasDeleteRequest struct {
Alias string `json:"alias"`
}
// DeleteAliasExperimental deletes a model alias via the experimental aliases API.
func (c *Client) DeleteAliasExperimental(ctx context.Context, req *AliasDeleteRequest) error {
return c.do(ctx, http.MethodDelete, "/api/experimental/aliases", req, nil)
}
+47 -14
View File
@@ -436,6 +436,7 @@ type ToolProperty struct {
Description string `json:"description,omitempty"`
Enum []any `json:"enum,omitempty"`
Properties *ToolPropertiesMap `json:"properties,omitempty"`
Required []string `json:"required,omitempty"`
}
// ToTypeScriptType converts a ToolProperty to a TypeScript type string
@@ -602,7 +603,7 @@ type Runner struct {
NumCtx int `json:"num_ctx,omitempty"`
NumBatch int `json:"num_batch,omitempty"`
NumGPU int `json:"num_gpu,omitempty"`
MainGPU int `json:"main_gpu,omitempty"`
MainGPU *int `json:"main_gpu,omitempty"`
UseMMap *bool `json:"use_mmap,omitempty"`
NumThread int `json:"num_thread,omitempty"`
}
@@ -801,6 +802,21 @@ type ListResponse struct {
Models []ListModelResponse `json:"models"`
}
// ModelRecommendationsResponse is the response from [Client.ModelRecommendationsExperimental].
type ModelRecommendationsResponse struct {
Recommendations []ModelRecommendation `json:"recommendations"`
}
// ModelRecommendation is a single recommendation entry in [ModelRecommendationsResponse].
type ModelRecommendation struct {
Model string `json:"model"`
Description string `json:"description"`
ContextLength int `json:"context_length,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
VRAMBytes int64 `json:"vram_bytes,omitempty"`
RequiredPlan string `json:"required_plan,omitempty"`
}
// ProcessResponse is the response from [Client.Process].
type ProcessResponse struct {
Models []ProcessModelResponse `json:"models"`
@@ -1030,14 +1046,25 @@ func (opts *Options) FromMap(m map[string]any) error {
}
field.Set(reflect.ValueOf(slice))
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
val, ok := val.(bool)
if !ok {
return fmt.Errorf("option %q must be of type boolean", key)
}
field.Set(reflect.ValueOf(&val))
} else {
case reflect.Int:
var i int
switch t := val.(type) {
case int64:
i = int(t)
case float64:
i = int(t)
default:
return fmt.Errorf("option %q must be of type integer", key)
}
field.Set(reflect.ValueOf(&i))
default:
return fmt.Errorf("unknown type loading config params: %v %v", field.Kind(), field.Type())
}
default:
@@ -1079,7 +1106,7 @@ func DefaultOptions() Options {
}
}
// ThinkValue represents a value that can be a boolean or a string ("high", "medium", "low")
// ThinkValue represents a value that can be a boolean or a string ("high", "medium", "low", "max")
type ThinkValue struct {
// Value can be a bool or string
Value interface{}
@@ -1095,7 +1122,7 @@ func (t *ThinkValue) IsValid() bool {
case bool:
return true
case string:
return v == "high" || v == "medium" || v == "low"
return v == "high" || v == "medium" || v == "low" || v == "max"
default:
return false
}
@@ -1129,8 +1156,8 @@ func (t *ThinkValue) Bool() bool {
case bool:
return v
case string:
// Any string value ("high", "medium", "low") means thinking is enabled
return v == "high" || v == "medium" || v == "low"
// Any string value ("high", "medium", "low", "max") means thinking is enabled
return v == "high" || v == "medium" || v == "low" || v == "max"
default:
return false
}
@@ -1168,14 +1195,14 @@ func (t *ThinkValue) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err == nil {
// Validate string values
if s != "high" && s != "medium" && s != "low" {
return fmt.Errorf("invalid think value: %q (must be \"high\", \"medium\", \"low\", true, or false)", s)
if s != "high" && s != "medium" && s != "low" && s != "max" {
return fmt.Errorf("invalid think value: %q (must be \"high\", \"medium\", \"low\", \"max\", true, or false)", s)
}
t.Value = s
return nil
}
return fmt.Errorf("think must be a boolean or string (\"high\", \"medium\", \"low\", true, or false)")
return fmt.Errorf("think must be a boolean or string (\"high\", \"medium\", \"low\", \"max\", true, or false)")
}
// MarshalJSON implements json.Marshaler
@@ -1278,14 +1305,20 @@ func FormatParams(params map[string][]string) (map[string]any, error) {
// TODO: only string slices are supported right now
out[key] = vals
case reflect.Pointer:
var b bool
if field.Type() == reflect.TypeOf(&b) {
switch field.Type().Elem().Kind() {
case reflect.Bool:
boolVal, err := strconv.ParseBool(vals[0])
if err != nil {
return nil, fmt.Errorf("invalid bool value %s", vals)
}
out[key] = &boolVal
} else {
case reflect.Int:
intVal, err := strconv.ParseInt(vals[0], 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid int value %s", vals)
}
out[key] = intVal
default:
return nil, fmt.Errorf("unknown type %s for %s", field.Kind(), key)
}
default:
+56
View File
@@ -20,6 +20,10 @@ func testPropsMap(m map[string]ToolProperty) *ToolPropertiesMap {
return props
}
func testIntPtr(v int) *int {
return &v
}
// testArgs creates ToolCallFunctionArguments from a map (convenience function for tests, order not preserved)
func testArgs(m map[string]any) ToolCallFunctionArguments {
args := NewToolCallFunctionArguments()
@@ -168,6 +172,47 @@ func TestUseMmapParsingFromJSON(t *testing.T) {
}
}
func TestMainGPUParsingFromJSON(t *testing.T) {
tests := []struct {
name string
req string
wantGPU *int
}{
{
name: "Undefined",
req: `{}`,
},
{
name: "Zero",
req: `{ "main_gpu": 0 }`,
wantGPU: testIntPtr(0),
},
{
name: "Nonzero",
req: `{ "main_gpu": 1 }`,
wantGPU: testIntPtr(1),
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var oMap map[string]any
err := json.Unmarshal([]byte(test.req), &oMap)
require.NoError(t, err)
opts := DefaultOptions()
err = opts.FromMap(oMap)
require.NoError(t, err)
if test.wantGPU == nil {
assert.Nil(t, opts.MainGPU)
} else if assert.NotNil(t, opts.MainGPU) {
assert.Equal(t, *test.wantGPU, *opts.MainGPU)
}
})
}
}
func TestUseMmapFormatParams(t *testing.T) {
tr := true
fa := false
@@ -232,6 +277,12 @@ func TestUseMmapFormatParams(t *testing.T) {
}
}
func TestMainGPUFormatParams(t *testing.T) {
resp, err := FormatParams(map[string][]string{"main_gpu": {"0"}})
require.NoError(t, err)
assert.Equal(t, int64(0), resp["main_gpu"])
}
func TestMessage_UnmarshalJSON(t *testing.T) {
tests := []struct {
input string
@@ -495,6 +546,11 @@ func TestThinking_UnmarshalJSON(t *testing.T) {
input: `{ "think": "low" }`,
expectedThinking: &ThinkValue{Value: "low"},
},
{
name: "string_max",
input: `{ "think": "max" }`,
expectedThinking: &ThinkValue{Value: "max"},
},
{
name: "invalid_string",
input: `{ "think": "invalid" }`,
-4
View File
@@ -157,10 +157,6 @@ func main() {
}
}
if u := os.Getenv("OLLAMA_UPDATE_URL"); u != "" {
updater.UpdateCheckURLBase = u
}
// Detect if this is a first start after an upgrade, in
// which case we need to do some cleanup
var skipMove bool
+1 -49
View File
@@ -90,9 +90,8 @@ DialogFontSize=12
[Files]
#if FileExists("..\dist\windows-ollama-app-amd64.exe")
Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#MyAppExeName}" ;Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('{#MyAppExeName}')
Source: "..\dist\windows-amd64\vc_redist.x64.exe"; DestDir: "{tmp}"; Check: not IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-amd64\ollama.exe"; DestDir: "{app}"; Check: not IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
Source: "..\dist\windows-amd64\lib\ollama\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
Source: "..\dist\windows-amd64\lib\ollama\*"; Excludes: "\mlx_*\*"; DestDir: "{app}\lib\ollama\"; Check: not IsArm64(); Flags: ignoreversion 64bit recursesubdirs
#endif
; For local development, rely on binary compatibility at runtime since we can't cross compile
@@ -103,7 +102,6 @@ Source: "..\dist\windows-ollama-app-amd64.exe"; DestDir: "{app}"; DestName: "{#M
#endif
#if FileExists("..\dist\windows-arm64\ollama.exe")
Source: "..\dist\windows-arm64\vc_redist.arm64.exe"; DestDir: "{tmp}"; Check: IsArm64() and vc_redist_needed(); Flags: deleteafterinstall
Source: "..\dist\windows-arm64\ollama.exe"; DestDir: "{app}"; Check: IsArm64(); Flags: ignoreversion 64bit; BeforeInstall: TaskKill('ollama.exe')
#endif
@@ -118,12 +116,6 @@ Name: "{userprograms}\{#MyAppName}"; Filename: "{app}\{#MyAppExeName}"; IconFile
Type: files; Name: "{%LOCALAPPDATA}\Ollama\updates"
[Run]
#if DirExists("..\dist\windows-arm64")
Filename: "{tmp}\vc_redist.arm64.exe"; Parameters: "/install /passive /norestart"; Check: IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
#if DirExists("..\dist\windows-amd64")
Filename: "{tmp}\vc_redist.x64.exe"; Parameters: "/install /passive /norestart"; Check: not IsArm64() and vc_redist_needed(); StatusMsg: "Installing VC++ Redistributables..."; Flags: waituntilterminated
#endif
Filename: "{cmd}"; Parameters: "/C set PATH={app};%PATH% & ""{app}\{#MyAppExeName}"""; Flags: postinstall nowait runhidden
[UninstallRun]
@@ -184,46 +176,6 @@ begin
Result := Pos(';' + ExpandConstant(Param) + ';', ';' + OrigPath + ';') = 0;
end;
{ --- VC Runtime libraries discovery code - Only install vc_redist if it isn't already installed ----- }
const VCRTL_MIN_V1 = 14;
const VCRTL_MIN_V2 = 40;
const VCRTL_MIN_V3 = 33807;
const VCRTL_MIN_V4 = 0;
// check if the minimum required vc redist is installed (by looking the registry)
function vc_redist_needed (): Boolean;
var
sRegKey: string;
v1: Cardinal;
v2: Cardinal;
v3: Cardinal;
v4: Cardinal;
begin
if (IsArm64()) then begin
sRegKey := 'SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\VC\Runtimes\arm64';
end else begin
sRegKey := 'SOFTWARE\Microsoft\VisualStudio\14.0\VC\Runtimes\x64';
end;
if (RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Major', v1) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Minor', v2) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'Bld', v3) and
RegQueryDWordValue (HKEY_LOCAL_MACHINE, sRegKey, 'RBld', v4)) then
begin
Log ('VC Redist version: ' + IntToStr (v1) +
'.' + IntToStr (v2) + '.' + IntToStr (v3) +
'.' + IntToStr (v4));
{ Version info was found. Return true if later or equal to our
minimal required version RTL_MIN_Vx }
Result := not (
(v1 > VCRTL_MIN_V1) or ((v1 = VCRTL_MIN_V1) and
((v2 > VCRTL_MIN_V2) or ((v2 = VCRTL_MIN_V2) and
((v3 > VCRTL_MIN_V3) or ((v3 = VCRTL_MIN_V3) and
(v4 >= VCRTL_MIN_V4)))))));
end
else
Result := TRUE;
end;
function GetDirSize(Path: String): Int64;
var
FindRec: TFindRec;
+23
View File
@@ -83,6 +83,29 @@ func resolvePath(name string) string {
return name
}
func ollamaServeArgs(args []string) bool {
if len(args) < 2 {
return false
}
switch strings.Trim(filepath.Base(args[0]), `"`) {
case "ollama", "ollama.exe":
default:
return false
}
for _, rawArg := range args[1:] {
arg := strings.Trim(rawArg, `"`)
if strings.HasPrefix(arg, "-") {
continue
}
return arg == "serve" || arg == "start"
}
return false
}
// cleanup checks the pid file for a running ollama process
// and shuts it down gracefully if it is running
func cleanup() error {
+57
View File
@@ -205,6 +205,63 @@ func TestServerCmdCloudSettingEnv(t *testing.T) {
}
}
func TestOllamaServeArgs(t *testing.T) {
tests := []struct {
name string
args []string
want bool
}{
{
name: "system ollama serve",
args: []string{"ollama", "serve"},
want: true,
},
{
name: "relative path ollama serve",
args: []string{"./ollama", "serve"},
want: true,
},
{
name: "serve after other flags",
args: []string{"./ollama", "--verbose", "serve"},
want: true,
},
{
name: "start alias",
args: []string{"ollama", "start"},
want: true,
},
{
name: "launch command",
args: []string{"ollama", "launch", "opencode"},
want: false,
},
{
name: "run command with model named serve",
args: []string{"ollama", "run", "serve"},
want: false,
},
{
name: "launch command with serve in passthrough args",
args: []string{"ollama", "launch", "codex", "--", "-p", "serve"},
want: false,
},
{
name: "different executable",
args: []string{"go", "run", "serve"},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ollamaServeArgs(tt.args); got != tt.want {
t.Fatalf("ollamaServeArgs(%v) = %v, want %v", tt.args, got, tt.want)
}
})
}
}
func TestGetInferenceInfo(t *testing.T) {
tests := []struct {
name string
+14 -1
View File
@@ -46,7 +46,17 @@ func terminated(pid int) (bool, error) {
return false, nil
}
// reapServers kills all ollama processes except our own
func ollamaServeProcess(pid int) bool {
output, err := exec.Command("ps", "-p", strconv.Itoa(pid), "-o", "args=").Output()
if err != nil {
slog.Debug("failed to inspect ollama process", "pid", pid, "err", err)
return false
}
return ollamaServeArgs(strings.Fields(strings.TrimSpace(string(output))))
}
// reapServers kills external ollama serve processes except our own.
func reapServers() error {
// Get our own PID to avoid killing ourselves
currentPID := os.Getpid()
@@ -82,6 +92,9 @@ func reapServers() error {
if pid == currentPID {
continue
}
if !ollamaServeProcess(pid) {
continue
}
proc, err := os.FindProcess(pid)
if err != nil {
+26 -1
View File
@@ -101,7 +101,29 @@ func terminated(pid int) (bool, error) {
return true, nil
}
// reapServers kills all ollama processes except our own
func ollamaServeProcess(pid int) bool {
cmd := exec.Command("wmic", "process", "where", fmt.Sprintf("ProcessId=%d", pid), "get", "CommandLine", "/value")
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
output, err := cmd.Output()
if err != nil {
slog.Debug("failed to inspect ollama process", "pid", pid, "err", err)
return false
}
for _, line := range strings.Split(string(output), "\n") {
line = strings.TrimSpace(line)
commandLine, ok := strings.CutPrefix(line, "CommandLine=")
if !ok {
continue
}
return ollamaServeArgs(strings.Fields(strings.ToLower(commandLine)))
}
return false
}
// reapServers kills external ollama serve processes except our own.
func reapServers() error {
// Get current process ID to avoid killing ourselves
currentPID := os.Getpid()
@@ -138,6 +160,9 @@ func reapServers() error {
if pid == currentPID {
continue
}
if !ollamaServeProcess(pid) {
continue
}
cmd := exec.Command("taskkill", "/F", "/PID", pidStr)
if err := cmd.Run(); err != nil {
+46 -5
View File
@@ -14,7 +14,7 @@ import (
// currentSchemaVersion defines the current database schema version.
// Increment this when making schema changes that require migrations.
const currentSchemaVersion = 15
const currentSchemaVersion = 16
// database wraps the SQLite connection.
// SQLite handles its own locking for concurrent access:
@@ -82,6 +82,7 @@ func (db *database) init() error {
websearch_enabled BOOLEAN NOT NULL DEFAULT 0,
selected_model TEXT NOT NULL DEFAULT '',
sidebar_open BOOLEAN NOT NULL DEFAULT 0,
last_home_view TEXT NOT NULL DEFAULT 'launch',
think_enabled BOOLEAN NOT NULL DEFAULT 0,
think_level TEXT NOT NULL DEFAULT '',
cloud_setting_migrated BOOLEAN NOT NULL DEFAULT 0,
@@ -264,6 +265,12 @@ func (db *database) migrate() error {
return fmt.Errorf("migrate v14 to v15: %w", err)
}
version = 15
case 15:
// add last_home_view column to settings table
if err := db.migrateV15ToV16(); err != nil {
return fmt.Errorf("migrate v15 to v16: %w", err)
}
version = 16
default:
// If we have a version we don't recognize, just set it to current
// This might happen during development
@@ -518,6 +525,21 @@ func (db *database) migrateV14ToV15() error {
return nil
}
// migrateV15ToV16 adds the last_home_view column to the settings table
func (db *database) migrateV15ToV16() error {
_, err := db.conn.Exec(`ALTER TABLE settings ADD COLUMN last_home_view TEXT NOT NULL DEFAULT 'launch'`)
if err != nil && !duplicateColumnError(err) {
return fmt.Errorf("add last_home_view column: %w", err)
}
_, err = db.conn.Exec(`UPDATE settings SET schema_version = 16`)
if err != nil {
return fmt.Errorf("update schema version: %w", err)
}
return nil
}
// cleanupOrphanedData removes orphaned records that may exist due to the foreign key bug
func (db *database) cleanupOrphanedData() error {
_, err := db.conn.Exec(`
@@ -1166,9 +1188,9 @@ func (db *database) getSettings() (Settings, error) {
var s Settings
err := db.conn.QueryRow(`
SELECT expose, survey, browser, models, agent, tools, working_dir, context_length, turbo_enabled, websearch_enabled, selected_model, sidebar_open, think_enabled, think_level, auto_update_enabled
SELECT expose, survey, browser, models, agent, tools, working_dir, context_length, turbo_enabled, websearch_enabled, selected_model, sidebar_open, last_home_view, think_enabled, think_level, auto_update_enabled
FROM settings
`).Scan(&s.Expose, &s.Survey, &s.Browser, &s.Models, &s.Agent, &s.Tools, &s.WorkingDir, &s.ContextLength, &s.TurboEnabled, &s.WebSearchEnabled, &s.SelectedModel, &s.SidebarOpen, &s.ThinkEnabled, &s.ThinkLevel, &s.AutoUpdateEnabled)
`).Scan(&s.Expose, &s.Survey, &s.Browser, &s.Models, &s.Agent, &s.Tools, &s.WorkingDir, &s.ContextLength, &s.TurboEnabled, &s.WebSearchEnabled, &s.SelectedModel, &s.SidebarOpen, &s.LastHomeView, &s.ThinkEnabled, &s.ThinkLevel, &s.AutoUpdateEnabled)
if err != nil {
return Settings{}, fmt.Errorf("get settings: %w", err)
}
@@ -1177,10 +1199,29 @@ func (db *database) getSettings() (Settings, error) {
}
func (db *database) setSettings(s Settings) error {
lastHomeView := strings.ToLower(strings.TrimSpace(s.LastHomeView))
validLaunchView := map[string]struct{}{
"launch": {},
"openclaw": {},
"claude": {},
"claude-desktop": {},
"hermes": {},
"codex": {},
"copilot": {},
"opencode": {},
"droid": {},
"pi": {},
}
if lastHomeView != "chat" {
if _, ok := validLaunchView[lastHomeView]; !ok {
lastHomeView = "launch"
}
}
_, err := db.conn.Exec(`
UPDATE settings
SET expose = ?, survey = ?, browser = ?, models = ?, agent = ?, tools = ?, working_dir = ?, context_length = ?, turbo_enabled = ?, websearch_enabled = ?, selected_model = ?, sidebar_open = ?, think_enabled = ?, think_level = ?, auto_update_enabled = ?
`, s.Expose, s.Survey, s.Browser, s.Models, s.Agent, s.Tools, s.WorkingDir, s.ContextLength, s.TurboEnabled, s.WebSearchEnabled, s.SelectedModel, s.SidebarOpen, s.ThinkEnabled, s.ThinkLevel, s.AutoUpdateEnabled)
SET expose = ?, survey = ?, browser = ?, models = ?, agent = ?, tools = ?, working_dir = ?, context_length = ?, turbo_enabled = ?, websearch_enabled = ?, selected_model = ?, sidebar_open = ?, last_home_view = ?, think_enabled = ?, think_level = ?, auto_update_enabled = ?
`, s.Expose, s.Survey, s.Browser, s.Models, s.Agent, s.Tools, s.WorkingDir, s.ContextLength, s.TurboEnabled, s.WebSearchEnabled, s.SelectedModel, s.SidebarOpen, lastHomeView, s.ThinkEnabled, s.ThinkLevel, s.AutoUpdateEnabled)
if err != nil {
return fmt.Errorf("set settings: %w", err)
}
+39
View File
@@ -135,6 +135,45 @@ func TestMigrationV13ToV14ContextLength(t *testing.T) {
}
}
func TestMigrationV15ToV16LastHomeViewDefaultsToLaunch(t *testing.T) {
tmpDir := t.TempDir()
dbPath := filepath.Join(tmpDir, "test.db")
db, err := newDatabase(dbPath)
if err != nil {
t.Fatalf("failed to create database: %v", err)
}
defer db.Close()
if _, err := db.conn.Exec(`
ALTER TABLE settings DROP COLUMN last_home_view;
UPDATE settings SET schema_version = 15;
`); err != nil {
t.Fatalf("failed to seed v15 settings row: %v", err)
}
if err := db.migrate(); err != nil {
t.Fatalf("migration from v15 to v16 failed: %v", err)
}
var lastHomeView string
if err := db.conn.QueryRow("SELECT last_home_view FROM settings").Scan(&lastHomeView); err != nil {
t.Fatalf("failed to read last_home_view: %v", err)
}
if lastHomeView != "launch" {
t.Fatalf("expected last_home_view to default to launch after migration, got %q", lastHomeView)
}
version, err := db.getSchemaVersion()
if err != nil {
t.Fatalf("failed to get schema version: %v", err)
}
if version != currentSchemaVersion {
t.Fatalf("expected schema version %d, got %d", currentSchemaVersion, version)
}
}
func TestChatDeletionWithCascade(t *testing.T) {
t.Run("chat deletion cascades to related messages", func(t *testing.T) {
tmpDir := t.TempDir()
+7
View File
@@ -167,6 +167,9 @@ type Settings struct {
// SidebarOpen indicates if the chat sidebar is open
SidebarOpen bool
// LastHomeView stores the preferred home route target ("chat" or integration name)
LastHomeView string
// AutoUpdateEnabled indicates if automatic updates should be downloaded
AutoUpdateEnabled bool
}
@@ -389,6 +392,10 @@ func (s *Store) Settings() (Settings, error) {
}
}
if settings.LastHomeView == "" {
settings.LastHomeView = "launch"
}
return settings, nil
}
+26
View File
@@ -81,6 +81,32 @@ func TestStore(t *testing.T) {
}
})
t.Run("settings default home view is launch", func(t *testing.T) {
loaded, err := s.Settings()
if err != nil {
t.Fatal(err)
}
if loaded.LastHomeView != "launch" {
t.Fatalf("expected default LastHomeView to be launch, got %q", loaded.LastHomeView)
}
})
t.Run("settings empty home view falls back to launch", func(t *testing.T) {
if err := s.SetSettings(Settings{LastHomeView: ""}); err != nil {
t.Fatal(err)
}
loaded, err := s.Settings()
if err != nil {
t.Fatal(err)
}
if loaded.LastHomeView != "launch" {
t.Fatalf("expected empty LastHomeView to fall back to launch, got %q", loaded.LastHomeView)
}
})
t.Run("window size", func(t *testing.T) {
if err := s.SetWindowSize(1024, 768); err != nil {
t.Fatal(err)
+4 -4
View File
@@ -414,6 +414,7 @@ export class Settings {
ThinkLevel: string;
SelectedModel: string;
SidebarOpen: boolean;
LastHomeView: string;
AutoUpdateEnabled: boolean;
constructor(source: any = {}) {
@@ -432,6 +433,7 @@ export class Settings {
this.ThinkLevel = source["ThinkLevel"];
this.SelectedModel = source["SelectedModel"];
this.SidebarOpen = source["SidebarOpen"];
this.LastHomeView = source["LastHomeView"];
this.AutoUpdateEnabled = source["AutoUpdateEnabled"];
}
}
@@ -550,14 +552,12 @@ export class Error {
}
}
export class ModelUpstreamResponse {
digest?: string;
pushTime: number;
stale: boolean;
error?: string;
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.digest = source["digest"];
this.pushTime = source["pushTime"];
this.stale = source["stale"];
this.error = source["error"];
}
}
@@ -0,0 +1 @@
<svg height="1em" style="flex:none;line-height:1" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><title>Claude Code</title><path clip-rule="evenodd" d="M20.998 10.949H24v3.102h-3v3.028h-1.487V20H18v-2.921h-1.487V20H15v-2.921H9V20H7.488v-2.921H6V20H4.487v-2.921H3V14.05H0V10.95h3V5h17.998v5.949zM6 10.949h1.488V8.102H6v2.847zm10.51 0H18V8.102h-1.49v2.847z" fill="#D97757" fill-rule="evenodd"></path></svg>

After

Width:  |  Height:  |  Size: 424 B

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated by Pixelmator Pro 3.6.17 -->
<svg width="1200" height="1200" viewBox="0 0 1200 1200" xmlns="http://www.w3.org/2000/svg">
<g id="g314">
<path id="path147" fill="#d97757" stroke="none" d="M 233.959793 800.214905 L 468.644287 668.536987 L 472.590637 657.100647 L 468.644287 650.738403 L 457.208069 650.738403 L 417.986633 648.322144 L 283.892639 644.69812 L 167.597321 639.865845 L 54.926208 633.825623 L 26.577238 627.785339 L 3.3e-05 592.751709 L 2.73832 575.27533 L 26.577238 559.248352 L 60.724873 562.228149 L 136.187973 567.382629 L 249.422867 575.194763 L 331.570496 580.026978 L 453.261841 592.671082 L 472.590637 592.671082 L 475.328857 584.859009 L 468.724915 580.026978 L 463.570557 575.194763 L 346.389313 495.785217 L 219.543671 411.865906 L 153.100723 363.543762 L 117.181267 339.060425 L 99.060455 316.107361 L 91.248367 266.01355 L 123.865784 230.093994 L 167.677887 233.073853 L 178.872513 236.053772 L 223.248367 270.201477 L 318.040283 343.570496 L 441.825592 434.738342 L 459.946411 449.798706 L 467.194672 444.64447 L 468.080597 441.020203 L 459.946411 427.409485 L 392.617493 305.718323 L 320.778564 181.932983 L 288.80542 130.630859 L 280.348999 99.865845 C 277.369171 87.221436 275.194641 76.590698 275.194641 63.624268 L 312.322174 13.20813 L 332.8591 6.604126 L 382.389313 13.20813 L 403.248352 31.328979 L 434.013519 101.71814 L 483.865753 212.537048 L 561.181274 363.221497 L 583.812134 407.919434 L 595.892639 449.315491 L 600.40271 461.959839 L 608.214783 461.959839 L 608.214783 454.711609 L 614.577271 369.825623 L 626.335632 265.61084 L 637.771851 131.516846 L 641.718201 93.745117 L 660.402832 48.483276 L 697.530334 24.000122 L 726.52356 37.852417 L 750.362549 72 L 747.060486 94.067139 L 732.886047 186.201416 L 705.100708 330.52356 L 686.979919 427.167847 L 697.530334 427.167847 L 709.61084 415.087341 L 758.496704 350.174561 L 840.644348 247.490051 L 876.885925 206.738342 L 919.167847 161.71814 L 946.308838 140.29541 L 997.61084 140.29541 L 1035.38269 196.429626 L 1018.469849 254.416199 L 965.637634 321.422852 L 921.825562 378.201538 L 859.006714 462.765259 L 819.785278 530.41626 L 823.409424 535.812073 L 832.75177 534.92627 L 974.657776 504.724915 L 1051.328979 490.872559 L 1142.818848 475.167786 L 1184.214844 494.496582 L 1188.724854 514.147644 L 1172.456421 554.335693 L 1074.604126 578.496765 L 959.838989 601.449829 L 788.939636 641.879272 L 786.845764 643.409485 L 789.261841 646.389343 L 866.255127 653.637634 L 899.194702 655.409424 L 979.812134 655.409424 L 1129.932861 666.604187 L 1169.154419 692.537109 L 1192.671265 724.268677 L 1188.724854 748.429688 L 1128.322144 779.194641 L 1046.818848 759.865845 L 856.590759 714.604126 L 791.355774 698.335754 L 782.335693 698.335754 L 782.335693 703.731567 L 836.69812 756.885986 L 936.322205 846.845581 L 1061.073975 962.81897 L 1067.436279 991.490112 L 1051.409424 1014.120911 L 1034.496704 1011.704712 L 924.885986 929.234924 L 882.604126 892.107544 L 786.845764 811.48999 L 780.483276 811.48999 L 780.483276 819.946289 L 802.550415 852.241699 L 919.087341 1027.409424 L 925.127625 1081.127686 L 916.671204 1098.604126 L 886.469849 1109.154419 L 853.288696 1103.114136 L 785.073914 1007.355835 L 714.684631 899.516785 L 657.906067 802.872498 L 650.979858 806.81897 L 617.476624 1167.704834 L 601.771851 1186.147705 L 565.530212 1200 L 535.328857 1177.046997 L 519.302124 1139.919556 L 535.328857 1066.550537 L 554.657776 970.792053 L 570.362488 894.68457 L 584.536926 800.134277 L 592.993347 768.724976 L 592.429626 766.630859 L 585.503479 767.516968 L 514.22821 865.369263 L 405.825531 1011.865906 L 320.053711 1103.677979 L 299.516815 1111.812256 L 263.919525 1093.369263 L 267.221497 1060.429688 L 287.114136 1031.114136 L 405.825531 880.107361 L 477.422913 786.52356 L 523.651062 732.483276 L 523.328918 724.671265 L 520.590698 724.671265 L 205.288605 929.395935 L 149.154434 936.644409 L 124.993355 914.01355 L 127.973183 876.885986 L 139.409409 864.80542 L 234.201385 799.570435 L 233.879227 799.8927 Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.0 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 320"><path fill="#fff" d="m297.06 130.97c7.26-21.79 4.76-45.66-6.85-65.48-17.46-30.4-52.56-46.04-86.84-38.68-15.25-17.18-37.16-26.95-60.13-26.81-35.04-.08-66.13 22.48-76.91 55.82-22.51 4.61-41.94 18.7-53.31 38.67-17.59 30.32-13.58 68.54 9.92 94.54-7.26 21.79-4.76 45.66 6.85 65.48 17.46 30.4 52.56 46.04 86.84 38.68 15.24 17.18 37.16 26.95 60.13 26.8 35.06.09 66.16-22.49 76.94-55.86 22.51-4.61 41.94-18.7 53.31-38.67 17.57-30.32 13.55-68.51-9.94-94.51zm-120.28 168.11c-14.03.02-27.62-4.89-38.39-13.88.49-.26 1.34-.73 1.89-1.07l63.72-36.8c3.26-1.85 5.26-5.32 5.24-9.07v-89.83l26.93 15.55c.29.14.48.42.52.74v74.39c-.04 33.08-26.83 59.9-59.91 59.97zm-128.84-55.03c-7.03-12.14-9.56-26.37-7.15-40.18.47.28 1.3.79 1.89 1.13l63.72 36.8c3.23 1.89 7.23 1.89 10.47 0l77.79-44.92v31.1c.02.32-.13.63-.38.83l-64.41 37.19c-28.69 16.52-65.33 6.7-81.92-21.95zm-16.77-139.09c7-12.16 18.05-21.46 31.21-26.29 0 .55-.03 1.52-.03 2.2v73.61c-.02 3.74 1.98 7.21 5.23 9.06l77.79 44.91-26.93 15.55c-.27.18-.61.21-.91.08l-64.42-37.22c-28.63-16.58-38.45-53.21-21.95-81.89zm221.26 51.49-77.79-44.92 26.93-15.54c.27-.18.61-.21.91-.08l64.42 37.19c28.68 16.57 38.51 53.26 21.94 81.94-7.01 12.14-18.05 21.44-31.2 26.28v-75.81c.03-3.74-1.96-7.2-5.2-9.06zm26.8-40.34c-.47-.29-1.3-.79-1.89-1.13l-63.72-36.8c-3.23-1.89-7.23-1.89-10.47 0l-77.79 44.92v-31.1c-.02-.32.13-.63.38-.83l64.41-37.16c28.69-16.55 65.37-6.7 81.91 22 6.99 12.12 9.52 26.31 7.15 40.1zm-168.51 55.43-26.94-15.55c-.29-.14-.48-.42-.52-.74v-74.39c.02-33.12 26.89-59.96 60.01-59.94 14.01 0 27.57 4.92 38.34 13.88-.49.26-1.33.73-1.89 1.07l-63.72 36.8c-3.26 1.85-5.26 5.31-5.24 9.06l-.04 89.79zm14.63-31.54 34.65-20.01 34.65 20v40.01l-34.65 20-34.65-20z"/></svg>

After

Width:  |  Height:  |  Size: 1.7 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 320"><path d="m297.06 130.97c7.26-21.79 4.76-45.66-6.85-65.48-17.46-30.4-52.56-46.04-86.84-38.68-15.25-17.18-37.16-26.95-60.13-26.81-35.04-.08-66.13 22.48-76.91 55.82-22.51 4.61-41.94 18.7-53.31 38.67-17.59 30.32-13.58 68.54 9.92 94.54-7.26 21.79-4.76 45.66 6.85 65.48 17.46 30.4 52.56 46.04 86.84 38.68 15.24 17.18 37.16 26.95 60.13 26.8 35.06.09 66.16-22.49 76.94-55.86 22.51-4.61 41.94-18.7 53.31-38.67 17.57-30.32 13.55-68.51-9.94-94.51zm-120.28 168.11c-14.03.02-27.62-4.89-38.39-13.88.49-.26 1.34-.73 1.89-1.07l63.72-36.8c3.26-1.85 5.26-5.32 5.24-9.07v-89.83l26.93 15.55c.29.14.48.42.52.74v74.39c-.04 33.08-26.83 59.9-59.91 59.97zm-128.84-55.03c-7.03-12.14-9.56-26.37-7.15-40.18.47.28 1.3.79 1.89 1.13l63.72 36.8c3.23 1.89 7.23 1.89 10.47 0l77.79-44.92v31.1c.02.32-.13.63-.38.83l-64.41 37.19c-28.69 16.52-65.33 6.7-81.92-21.95zm-16.77-139.09c7-12.16 18.05-21.46 31.21-26.29 0 .55-.03 1.52-.03 2.2v73.61c-.02 3.74 1.98 7.21 5.23 9.06l77.79 44.91-26.93 15.55c-.27.18-.61.21-.91.08l-64.42-37.22c-28.63-16.58-38.45-53.21-21.95-81.89zm221.26 51.49-77.79-44.92 26.93-15.54c.27-.18.61-.21.91-.08l64.42 37.19c28.68 16.57 38.51 53.26 21.94 81.94-7.01 12.14-18.05 21.44-31.2 26.28v-75.81c.03-3.74-1.96-7.2-5.2-9.06zm26.8-40.34c-.47-.29-1.3-.79-1.89-1.13l-63.72-36.8c-3.23-1.89-7.23-1.89-10.47 0l-77.79 44.92v-31.1c-.02-.32.13-.63.38-.83l64.41-37.16c28.69-16.55 65.37-6.7 81.91 22 6.99 12.12 9.52 26.31 7.15 40.1zm-168.51 55.43-26.94-15.55c-.29-.14-.48-.42-.52-.74v-74.39c.02-33.12 26.89-59.96 60.01-59.94 14.01 0 27.57 4.92 38.34 13.88-.49.26-1.33.73-1.89 1.07l-63.72 36.8c-3.26 1.85-5.26 5.31-5.24 9.06l-.04 89.79zm14.63-31.54 34.65-20.01 34.65 20v40.01l-34.65 20-34.65-20z"/></svg>

After

Width:  |  Height:  |  Size: 1.7 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="currentColor" fill-rule="evenodd" style="flex:none;line-height:1" viewBox="0 2.5 24 19"><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z" fill="white"/></svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="currentColor" fill-rule="evenodd" style="flex:none;line-height:1" viewBox="0 2.5 24 19"><path d="M19.245 5.364c1.322 1.36 1.877 3.216 2.11 5.817.622 0 1.2.135 1.592.654l.73.964c.21.278.323.61.323.955v2.62c0 .339-.173.669-.453.868C20.239 19.602 16.157 21.5 12 21.5c-4.6 0-9.205-2.583-11.547-4.258-.28-.2-.452-.53-.453-.868v-2.62c0-.345.113-.679.321-.956l.73-.963c.392-.517.974-.654 1.593-.654l.029-.297c.25-2.446.81-4.213 2.082-5.52 2.461-2.54 5.71-2.851 7.146-2.864h.198c1.436.013 4.685.323 7.146 2.864zm-7.244 4.328c-.284 0-.613.016-.962.05-.123.447-.305.85-.57 1.108-1.05 1.023-2.316 1.18-2.994 1.18-.638 0-1.306-.13-1.851-.464-.516.165-1.012.403-1.044.996a65.882 65.882 0 00-.063 2.884l-.002.48c-.002.563-.005 1.126-.013 1.69.002.326.204.63.51.765 2.482 1.102 4.83 1.657 6.99 1.657 2.156 0 4.504-.555 6.985-1.657a.854.854 0 00.51-.766c.03-1.682.006-3.372-.076-5.053-.031-.596-.528-.83-1.046-.996-.546.333-1.212.464-1.85.464-.677 0-1.942-.157-2.993-1.18-.266-.258-.447-.661-.57-1.108-.32-.032-.64-.049-.96-.05zm-2.525 4.013c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zm5 0c.539 0 .976.426.976.95v1.753c0 .525-.437.95-.976.95a.964.964 0 01-.976-.95v-1.752c0-.525.437-.951.976-.951zM7.635 5.087c-1.05.102-1.935.438-2.385.906-.975 1.037-.765 3.668-.21 4.224.405.394 1.17.657 1.995.657h.09c.649-.013 1.785-.176 2.73-1.11.435-.41.705-1.433.675-2.47-.03-.834-.27-1.52-.63-1.813-.39-.336-1.275-.482-2.265-.394zm6.465.394c-.36.292-.6.98-.63 1.813-.03 1.037.24 2.06.675 2.47.968.957 2.136 1.104 2.776 1.11h.044c.825 0 1.59-.263 1.995-.657.555-.556.765-3.187-.21-4.224-.45-.468-1.335-.804-2.385-.906-.99-.088-1.875.058-2.265.394zM12 7.615c-.24 0-.525.015-.84.044.03.16.045.336.06.526l-.001.159a2.94 2.94 0 01-.014.25c.225-.022.425-.027.612-.028h.366c.187 0 .387.006.612.028-.015-.146-.015-.277-.015-.409.015-.19.03-.365.06-.526a9.29 9.29 0 00-.84-.044z"/></svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

File diff suppressed because one or more lines are too long.

After

Width:  |  Height:  |  Size: 6.2 KiB

@@ -0,0 +1,181 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="1000" height="1000" viewBox="0 0 1000 1000"><circle cx="500.0" cy="500.0" r="500.0" fill="white"/><g transform="translate(100.0 100.0) scale(0.8333333333333334)"><g transform="translate(0.000000,960.000000) scale(0.100000,-0.100000)"
fill="black" stroke="none">
<path d="M4485 9589 c-248 -27 -432 -60 -730 -130 -458 -108 -798 -230 -1207
-435 -533 -267 -1072 -675 -1358 -1030 -205 -255 -442 -748 -535 -1114 -108
-426 -97 -870 29 -1160 73 -169 236 -369 381 -467 139 -94 425 -206 425 -167
0 3 -26 32 -58 63 -74 71 -147 182 -184 278 -16 41 -27 77 -25 79 2 3 31 -28
63 -68 91 -114 153 -177 231 -235 40 -29 77 -62 83 -73 7 -13 4 -85 -10 -242
-10 -123 -24 -281 -30 -353 -6 -71 -29 -296 -51 -500 -135 -1262 -202 -1568
-378 -1733 -69 -64 -105 -77 -216 -77 -132 0 -188 20 -271 97 -110 103 -165
248 -167 438 -1 123 12 191 60 318 20 51 33 95 29 99 -10 10 -92 -74 -136
-137 -153 -223 -204 -504 -146 -790 19 -91 97 -252 161 -333 112 -141 316
-237 504 -237 180 0 419 118 591 290 81 81 107 115 196 255 50 78 54 56 13
-71 -107 -337 -343 -577 -613 -625 -213 -39 -544 99 -707 295 -93 111 -133
199 -173 381 -34 153 -34 154 -46 135 -15 -23 -12 -194 5 -305 42 -280 149
-488 327 -636 133 -111 287 -195 465 -254 62 -20 115 -40 117 -44 3 -4 12 -43
21 -87 44 -222 199 -385 416 -438 96 -24 265 -21 359 5 138 38 281 148 388
297 39 55 48 63 50 45 4 -27 -38 -185 -78 -294 -80 -217 -176 -374 -312 -515
-54 -55 -98 -104 -98 -107 0 -4 245 -7 545 -7 l544 0 126 66 c69 36 130 64
135 62 6 -2 -10 -28 -34 -59 -46 -59 -48 -69 -10 -69 17 0 32 15 58 59 36 59
51 71 87 71 23 0 25 -27 4 -77 -8 -19 -15 -39 -15 -44 0 -12 447 -11 470 1 10
5 85 87 168 182 257 297 400 415 317 263 -16 -30 -26 -57 -23 -60 11 -12 210
100 383 215 115 76 226 143 375 225 58 32 178 100 266 151 150 87 244 132 244
117 0 -4 -78 -86 -174 -182 -207 -208 -246 -254 -334 -386 -48 -71 -122 -157
-259 -298 -117 -120 -193 -206 -193 -217 0 -19 8 -20 115 -20 101 0 116 2 122
18 19 54 112 249 156 327 124 221 283 436 369 502 87 67 225 152 337 209 103
51 297 117 384 130 38 6 29 -2 -92 -75 -74 -45 -170 -107 -214 -139 -106 -76
-247 -225 -332 -351 -67 -99 -70 -102 -79 -77 -6 14 -13 26 -18 26 -11 0 -57
-75 -94 -155 -63 -136 -124 -376 -103 -401 15 -18 152 -19 166 -2 6 7 18 36
27 63 23 72 66 131 217 301 302 339 430 464 590 577 148 104 233 128 520 148
205 14 255 9 447 -38 118 -29 183 -65 298 -163 229 -195 538 -577 579 -715 24
-83 71 -117 109 -79 9 8 16 27 16 42 0 62 -40 212 -73 277 -158 313 -551 635
-922 755 -127 41 -142 55 -52 46 117 -11 231 -35 321 -65 158 -54 259 -123
405 -277 164 -174 283 -379 322 -556 11 -51 25 -113 32 -137 36 -128 148 -81
117 49 -14 60 -7 106 27 177 36 74 62 94 224 179 254 133 425 260 561 417 272
315 403 732 358 1138 -24 218 -84 401 -179 545 -65 98 -155 203 -166 192 -3
-3 7 -37 23 -75 136 -309 136 -725 2 -1063 -130 -330 -441 -760 -633 -879 -60
-37 -60 -20 2 54 478 577 578 1337 315 2390 -25 99 -86 326 -136 505 -169 611
-386 1539 -494 2109 -161 857 -200 998 -442 1606 -161 405 -321 692 -529 950
-93 114 -322 343 -433 431 -203 160 -497 332 -737 428 -293 118 -702 220 -978
246 -129 12 -406 11 -520 -1z m-267 -214 c301 -47 596 -191 852 -419 85 -76
266 -270 345 -371 294 -376 520 -873 640 -1409 45 -199 45 -222 3 -244 -18 -9
-45 -30 -61 -45 -25 -23 -28 -33 -23 -60 9 -44 21 -54 121 -98 91 -40 225
-124 225 -140 0 -9 -33 5 -211 89 -126 60 -159 94 -167 173 -5 50 14 89 43 89
26 0 65 49 65 81 0 52 -22 110 -48 127 -23 15 -37 14 -186 -7 -172 -25 -210
-36 -240 -69 -23 -27 -31 -99 -14 -131 16 -29 30 -37 85 -51 26 -7 44 -17 48
-29 10 -32 -15 -153 -41 -197 -27 -49 -131 -161 -139 -152 -9 9 36 77 85 126
23 24 48 60 56 79 13 31 13 38 -3 71 -10 20 -29 42 -43 48 -14 7 -37 18 -52
24 -33 15 -42 39 -63 155 -24 133 -90 393 -135 532 -119 367 -287 686 -483
919 -175 208 -434 427 -659 557 -229 131 -498 197 -804 197 -136 0 -185 -4
-313 -26 -87 -14 -31 14 137 70 186 61 289 89 437 117 120 23 376 20 543 -6z
m2913 -962 c50 -53 113 -138 198 -268 76 -116 60 -104 -41 32 -34 46 -63 81
-66 79 -2 -2 2 -30 9 -63 9 -44 9 -86 1 -171 -6 -63 -13 -116 -16 -119 -3 -4
-15 2 -26 12 -32 29 -34 12 -5 -57 50 -118 66 -160 62 -164 -2 -2 -29 34 -61
81 -31 47 -61 83 -66 80 -6 -4 -7 -24 -4 -47 7 -39 6 -40 -14 -27 -12 7 -25
10 -29 5 -17 -17 -4 -106 31 -210 20 -60 35 -111 33 -112 -2 -2 -30 43 -62 99
-60 102 -173 233 -182 210 -2 -7 26 -82 62 -168 37 -85 65 -160 63 -166 -2 -6
-43 66 -91 160 -63 121 -93 171 -106 171 -9 0 -37 -21 -61 -46 -55 -56 -56
-56 -248 20 -78 31 -156 59 -172 63 l-30 6 24 -54 c13 -30 51 -114 85 -188 34
-73 60 -135 58 -137 -2 -3 -21 24 -42 58 -56 91 -85 128 -102 128 -13 0 -14
-8 -9 -42 l7 -42 -83 80 c-109 106 -163 133 -260 126 -35 -3 -38 0 -53 34 -9
21 -13 44 -11 51 7 17 58 16 147 -2 101 -21 104 -17 92 106 -6 52 -7 98 -3
102 4 5 25 -20 47 -55 22 -35 47 -68 54 -75 18 -14 278 -83 316 -83 23 0 42
13 86 59 61 64 64 72 42 111 -19 33 -19 54 0 46 11 -4 23 6 37 30 l21 35 66
-3 66 -3 -2 29 c-1 16 -25 63 -52 105 -28 41 -49 77 -47 78 2 2 47 -42 101
-97 75 -76 105 -100 125 -100 14 0 35 -7 47 -15 20 -14 22 -14 27 2 3 10 6 72
7 138 2 102 -1 128 -19 175 -12 30 -22 56 -22 58 0 9 40 -22 71 -55z m-2642
-139 c29 -35 60 -64 67 -64 8 0 52 27 97 61 l82 60 47 -51 c59 -63 161 -218
218 -327 23 -46 48 -83 55 -83 8 0 26 5 41 11 49 18 73 4 104 -64 33 -70 48
-127 33 -127 -6 0 -32 9 -58 21 -28 12 -56 18 -71 15 -21 -6 -27 1 -55 56 -67
132 -208 340 -244 361 -10 5 -19 -1 -29 -22 -29 -55 -35 -106 -20 -183 8 -40
14 -74 14 -75 0 -1 -12 2 -26 8 l-27 10 7 -62 c6 -61 6 -62 -14 -44 -15 14
-31 17 -72 13 -58 -6 -88 -32 -88 -76 l0 -26 -29 34 c-29 35 -30 35 -122 38
-52 2 -99 8 -106 14 -15 12 -83 132 -83 146 0 6 23 39 50 73 56 70 68 99 52
134 -12 27 -15 25 81 46 65 14 68 21 42 105 -26 85 -18 85 54 -2z m-397 -294
c84 -126 196 -352 237 -475 33 -99 28 -115 -23 -66 -45 44 -55 29 -49 -77 5
-95 -4 -97 -33 -7 -22 67 -52 125 -64 125 -6 0 -10 -39 -11 -92 0 -51 -4 -101
-8 -111 -9 -23 -10 -22 -85 101 -32 50 -62 92 -68 92 -7 0 -9 -22 -4 -70 6
-74 -3 -90 -24 -40 -21 50 -33 54 -87 30 -26 -11 -57 -30 -69 -41 -20 -19 -21
-19 -75 10 -30 17 -91 41 -137 54 -46 13 -89 30 -97 38 -16 16 -45 142 -36
157 3 5 58 33 121 61 l114 51 21 -26 21 -26 49 39 c51 40 69 66 80 116 5 23
10 28 32 25 19 -2 28 -11 37 -38 37 -115 42 114 6 250 -45 171 -47 164 22 90
33 -36 92 -112 130 -170z m-1789 73 c-3 -10 -32 -77 -63 -148 -48 -109 -137
-340 -242 -628 -11 -32 -22 -56 -24 -54 -2 2 -9 32 -14 68 -24 141 -20 131
-47 124 -13 -3 -50 -18 -81 -33 -43 -20 -62 -37 -79 -67 -32 -59 -40 -65 -88
-65 -55 0 -56 6 -16 106 29 75 176 339 194 351 12 7 14 14 -47 -125 -25 -56
-46 -113 -46 -127 0 -25 0 -25 35 -11 109 46 179 120 301 316 135 217 241 360
217 293z m-869 -35 c-4 -7 -33 -49 -64 -93 -140 -200 -268 -431 -368 -665
-114 -268 -153 -314 -74 -86 41 115 44 129 29 137 -28 16 -39 80 -22 128 27
77 98 202 139 246 58 61 355 345 362 345 3 0 2 -6 -2 -12z m1415 -533 l1 -130
-23 39 c-26 47 -41 51 -45 14 -4 -36 -18 -35 -37 1 -8 17 -19 32 -25 36 -5 3
-60 -10 -122 -30 -76 -25 -130 -36 -165 -36 -29 1 -82 -5 -118 -14 -99 -23
-109 -21 -149 29 -20 23 -36 50 -36 58 0 12 55 182 75 231 11 27 38 21 147
-34 76 -38 108 -49 130 -45 21 4 28 2 28 -9 0 -10 11 -15 34 -15 45 0 153 34
177 56 10 9 35 69 55 133 56 180 58 180 65 1 4 -85 7 -213 8 -285z m-1394 272
c-31 -55 -32 -83 -2 -91 47 -12 65 -6 97 34 18 23 34 39 36 38 2 -2 -20 -48
-48 -102 l-50 -99 33 7 c84 17 149 19 149 5 0 -8 -37 -104 -82 -213 -74 -177
-83 -195 -86 -162 -4 50 -26 56 -66 17 -17 -17 -35 -31 -39 -31 -4 0 -7 18 -7
40 0 28 -6 43 -18 51 -15 10 -18 21 -14 65 8 93 -18 69 -89 -80 -35 -74 -65
-133 -67 -131 -3 2 5 35 17 74 11 38 21 74 21 80 0 6 -35 11 -87 13 l-88 3 3
30 c4 42 162 364 187 381 11 8 44 14 76 14 55 0 59 2 93 42 20 23 36 45 36 50
0 4 5 8 10 8 6 0 -1 -20 -15 -43z m1710 6 c120 -17 143 -19 164 -12 10 3 21
-17 37 -67 24 -77 75 -306 69 -312 -2 -2 -21 14 -43 37 l-39 41 -145 0 c-128
0 -148 2 -170 19 -16 13 -29 17 -39 10 -8 -5 -17 -9 -20 -9 -10 0 -66 178 -74
232 -4 25 -4 56 0 68 7 21 11 22 79 16 39 -4 121 -14 181 -23z m-79 -988 c74
-190 129 -303 247 -514 76 -136 79 -145 62 -157 -24 -17 -76 -18 -98 -1 -21
16 -126 237 -165 347 -30 87 -143 516 -141 541 1 19 17 -19 95 -216z m3644 61
c64 -163 185 -534 301 -926 66 -223 147 -490 179 -595 175 -566 250 -858 321
-1240 22 -121 43 -231 46 -245 4 -18 3 -22 -6 -15 -6 6 -22 71 -36 145 -58
313 -100 487 -200 820 -40 135 -94 317 -120 405 -161 550 -413 1387 -465 1540
-55 165 -67 205 -56 194 2 -2 18 -39 36 -83z m-4395 -812 c61 -60 129 -118
149 -128 46 -22 69 -19 238 25 70 18 130 30 133 27 3 -3 -19 -42 -49 -87 -53
-78 -75 -125 -63 -137 14 -14 111 32 205 96 57 38 136 85 176 104 84 40 299
116 327 116 12 0 37 -24 66 -65 25 -36 54 -67 62 -69 9 -2 178 -1 376 2 l360
7 10 70 c10 68 10 69 22 40 7 -16 17 -49 23 -73 14 -53 18 -55 187 -82 178
-28 191 -33 200 -78 12 -57 8 -612 -6 -742 -17 -170 -53 -395 -141 -880 -206
-1142 -248 -1540 -194 -1863 8 -49 12 -97 9 -107 -8 -24 -195 -200 -213 -200
-7 0 -21 14 -30 31 -140 256 -353 528 -467 594 -66 39 -95 39 -361 1 -137 -19
-303 -40 -368 -47 -128 -12 -307 -7 -368 11 -54 16 -140 78 -185 132 -41 51
-348 610 -438 801 -105 220 -178 478 -191 667 -7 97 11 328 25 343 5 5 23 -17
41 -49 18 -32 72 -97 125 -148 54 -53 97 -104 101 -120 12 -50 -1 -119 -34
-170 -22 -33 -32 -62 -32 -88 0 -45 31 -126 70 -182 23 -35 28 -50 23 -82 -3
-24 4 -70 17 -119 18 -68 28 -87 68 -128 59 -60 118 -101 132 -92 6 4 10 18 8
32 -3 22 3 28 48 44 28 11 59 28 70 40 l18 20 -107 -7 c-118 -8 -146 0 -166
43 -17 37 -14 52 18 84 33 33 78 41 66 12 -14 -31 -16 -77 -5 -93 9 -13 14
-11 36 14 21 24 25 37 21 68 -5 37 -4 38 36 49 56 15 192 6 242 -15 l40 -17
-27 -20 c-66 -49 -1 -50 120 -2 80 31 92 40 92 64 0 33 -30 52 -72 45 -33 -5
-51 1 -140 49 -226 121 -267 139 -327 139 -31 1 -68 -3 -83 -8 -24 -7 -32 -3
-62 30 -62 67 -63 76 -30 145 53 111 38 151 -110 308 -88 93 -131 162 -142
230 -11 67 0 84 99 164 142 113 222 232 261 384 46 178 -18 329 -188 441 -74
48 -75 49 -51 60 35 16 31 28 -20 64 -31 22 -41 34 -32 40 21 14 168 59 252
77 67 15 80 21 83 39 2 12 -33 92 -82 187 -100 193 -117 263 -35 144 28 -41
102 -124 164 -185z m-430 -364 c-13 -21 19 -51 100 -97 78 -43 142 -93 119
-93 -5 0 -34 7 -64 15 -69 19 -148 19 -180 0 -24 -14 -24 -14 21 -15 63 0 238
-37 267 -56 24 -16 42 -52 42 -85 0 -17 -8 -14 -57 24 -92 71 -138 91 -213 90
-36 0 -85 -8 -109 -17 -55 -20 -64 -14 -103 71 -37 82 -36 104 4 127 60 36
190 63 173 36z m4328 -154 c76 -37 148 -103 127 -116 -27 -17 -107 -10 -174
15 -91 34 -212 35 -310 1 -55 -19 -76 -22 -100 -14 -17 5 -37 12 -45 14 -17 6
18 45 66 75 76 47 131 59 258 57 109 -3 125 -6 178 -32z m-4293 -282 c3 -28
11 -58 17 -66 7 -8 12 -30 12 -49 -1 -40 10 -72 46 -128 33 -53 32 -67 -5 -80
-78 -27 -118 37 -133 210 -10 105 -9 120 7 144 27 42 50 29 56 -31z m-53 -438
c-4 -9 -11 -16 -17 -16 -11 0 -14 33 -3 44 11 10 26 -11 20 -28z m449 -922
c60 -20 62 -23 44 -34 -23 -15 -104 -12 -126 5 -19 15 -19 15 0 30 24 18 22
19 82 -1z m5843 -211 c82 -179 180 -472 218 -653 34 -160 38 -387 10 -517 -37
-172 -107 -345 -191 -471 -81 -122 -215 -266 -232 -249 -2 2 10 37 27 78 188
447 261 1077 185 1584 -15 98 -31 196 -35 217 -10 44 0 50 18 11z m-2689 -313
c25 -333 24 -319 22 -342 -1 -10 -66 -56 -164 -115 -175 -106 -203 -121 -196
-101 3 7 26 81 53 163 39 124 214 633 241 699 15 37 22 -12 44 -304z m110
-832 c37 -172 37 -173 -56 -257 -80 -73 -143 -103 -230 -109 -105 -7 -132 9
-183 108 -53 101 -53 132 -3 177 21 19 126 88 233 153 182 111 194 117 201 97
3 -12 20 -88 38 -169z m-1046 -650 c67 -151 141 -236 331 -383 138 -106 239
-173 309 -204 42 -18 47 -23 36 -36 -7 -9 -110 -81 -229 -160 -178 -118 -251
-161 -416 -238 -110 -51 -250 -117 -312 -147 -62 -29 -118 -50 -126 -47 -7 3
-24 34 -36 69 -28 77 -74 158 -252 445 -75 122 -140 231 -144 242 -6 19 -2 21
42 21 63 0 171 24 230 50 35 15 99 72 238 209 182 180 271 261 285 261 4 0 24
-37 44 -82z m-2387 -88 c-10 -81 -64 -284 -102 -378 -55 -136 -119 -238 -204
-323 -108 -107 -162 -132 -306 -137 -109 -4 -111 -3 -175 30 -42 22 -76 48
-98 78 l-35 45 114 7 c338 22 478 105 645 383 29 50 70 131 89 180 49 124 67
165 73 165 3 0 2 -22 -1 -50z m1647 -1402 c-54 -78 -300 -358 -315 -358 -16 0
-10 15 29 76 113 173 323 408 330 370 2 -10 -18 -49 -44 -88z"/>
<path d="M3229 5845 c-108 -15 -150 -30 -198 -71 -49 -41 -111 -123 -111 -146
0 -18 5 -20 40 -15 22 3 40 3 40 1 0 -2 -9 -26 -20 -53 -12 -32 -16 -52 -9
-56 9 -6 7 -23 -8 -62 -3 -8 4 -13 20 -13 16 0 26 -7 30 -20 8 -30 33 -24 48
12 15 37 122 148 142 148 12 0 11 -10 0 -56 -17 -66 -10 -118 17 -137 15 -11
19 -21 14 -44 -5 -25 2 -39 44 -90 28 -33 66 -67 85 -76 39 -19 112 -22 152
-7 39 15 108 74 140 121 27 38 28 41 13 72 -15 32 -15 34 8 54 13 12 24 33 24
47 l0 25 38 -17 c58 -26 111 -62 122 -81 6 -13 3 -29 -11 -57 -98 -186 -404
-264 -685 -174 -101 32 -143 37 -162 18 -21 -21 -13 -28 31 -28 49 0 82 -19
91 -53 5 -20 11 -23 31 -19 14 2 31 0 38 -6 17 -14 145 -34 168 -27 11 4 26 1
34 -5 9 -7 23 -9 37 -4 13 5 41 9 63 11 22 1 51 3 65 4 14 1 37 -1 52 -5 21
-6 27 -4 33 13 5 18 14 21 55 21 43 0 49 3 52 23 3 19 10 22 51 25 49 3 53 7
37 32 -11 17 4 30 38 30 15 0 22 6 22 19 0 14 16 27 55 45 40 18 56 31 61 51
3 14 16 32 27 40 18 13 20 18 10 34 -11 17 -8 21 25 35 100 44 116 63 55 68
-33 3 -38 6 -36 26 3 22 0 22 -45 16 -44 -6 -49 -4 -83 29 -58 56 -354 199
-469 227 -116 28 -147 43 -70 36 30 -3 108 -23 173 -45 64 -22 117 -36 117
-31 0 13 -23 24 -160 75 -142 53 -197 60 -331 40z"/>
<path d="M3027 5303 c-3 -5 -2 -15 2 -22 7 -10 10 -10 16 -1 4 6 3 16 -3 22
-5 5 -12 6 -15 1z"/>
<path d="M2180 4462 c0 -11 136 -122 149 -122 19 0 12 46 -11 75 -12 15 -36
34 -54 41 -37 15 -84 19 -84 6z"/>
</g></g></svg>

After

Width:  |  Height:  |  Size: 13 KiB

+242
View File
@@ -0,0 +1,242 @@
<svg version="1.2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 500 500" width="500" height="500">
<style>
.s0 { fill: #f6f4f4 }
.s1 { fill: #0b0303 }
.s2 { fill: #ef0011 }
.s3 { fill: #f3e2e2 }
.s4 { fill: #f00212 }
.s5 { fill: #ba000d }
.s6 { fill: #faf1f1 }
.s7 { fill: #0b0100 }
.s8 { fill: #fbedee }
.s9 { fill: #faeaea }
.s10 { fill: #ab797d }
.s11 { fill: #f8eaea }
.s12 { fill: #902021 }
.s13 { fill: #f9eeee }
.s14 { fill: #f6ecec }
.s15 { fill: #080201 }
.s16 { fill: #150100 }
.s17 { fill: #f2e7e7 }
.s18 { fill: #fbe7e8 }
.s19 { fill: #060101 }
.s20 { fill: #f5e7e7 }
.s21 { fill: #fa999e }
.s22 { fill: #c46064 }
.s23 { fill: #180300 }
.s24 { fill: #f6dcdd }
.s25 { fill: #f2e6e6 }
.s26 { fill: #110200 }
.s27 { fill: #eb0011 }
.s28 { fill: #e20010 }
.s29 { fill: #ea0011 }
.s30 { fill: #760007 }
.s31 { fill: #f00514 }
.s32 { fill: #fcebeb }
.s33 { fill: #ecd6d6 }
.s34 { fill: #f5e3e3 }
.s35 { fill: #f5e4e4 }
.s36 { fill: #faf6f6 }
.s37 { fill: #e50010 }
.s38 { fill: #d5000f }
.s39 { fill: #f2e2e3 }
.s40 { fill: #ef1018 }
.s41 { fill: #f4e8e9 }
.s42 { fill: #ef0513 }
.s43 { fill: #f5e5e5 }
.s44 { fill: #f00413 }
.s45 { fill: #f4e9ea }
.s46 { fill: #ed0011 }
.s47 { fill: #e80011 }
.s48 { fill: #e60613 }
.s49 { fill: #f0d6d6 }
.s50 { fill: #fca9ac }
.s51 { fill: #9c000c }
.s52 { fill: #73393b }
</style>
<g>
<path fill-rule="evenodd" class="s0" d="m166.5 52.5q3.5 0 7 0 2.75 2.99 1.5 7-21.27 45.61-20.5 96 39.99 2.76 72 26.5 7.87 6.86 13.5 15.5 42.88-56.39 103.5-92.5 47.35-25.46 101-25 14.52 0.38 23.5 11.5 3.19 7.74 2 16-1.81 7.18-4.5 14-1 0-1 1-5.04 6.05-9 13-1 0-1 1 0 0.5 0 1-12.42 12.15-28.5 19-6.02 36.27-41.5 45-0.83 2.75 0 5 19.02-12.85 41.5-9 10.85-8.09 23.5-13 15.01-6.37 31-2.5 14.09 7.43 14 23.5-2.83 23.25-15.5 43-6.42 9.92-14 19-10.04 8.8-19.5 18-72.02 48.88-156.5 27-19.63 9.6-41.5 10.5-4.59 1.27-9 3 2 1 4 2 20.09-1.11 35 12 25.46 6.95 37.5 30.5 1.26 5.69-1 11-3.38 3.79-7.5 6.5 5.74 10.07 1.5 20.5-7.55 7.47-17.5 3.5-11.01-5.34-22.5-9.5-18.26 10-38.5 13-15.5 0-31 0-26.62-4.54-51-17-4.17 1.33-8 3.5-7.23 5.87-15 11-8.62 2.58-13.5-4.5-1.82 2.32-4.5 3.5-6.06 2.24-12 3.5-7.5 0-15 0-27.42-2.56-50-18.5-18-17.25-23-41.5 0-11.5 0-23 4.12-22.7 25-33 6.95-16.67 22-26.5-20.39-20.8-14.5-49.5 7.01-26.98 28.5-44.5 7.56-5.27 15-10.5-13.09-30.88-7.5-64 3.16-15.57 14.5-26.5 6.85-2.48 8 4.5-6.59 39.53 11 75.5 7.99-0.49 16-2 2.42-34.57 14.5-67.5 8.51-22.23 27.5-36z"/>
</g>
<g>
<path fill-rule="evenodd" class="s1" d="m113.5 401.5q0.48-5.1-1-10-0.91 0.19-1 1-2.46 1.74-5 3.5 5.65 9.54-5 13-32.21 5.55-61-10-32.89-23.11-29.5-63.5 2.96-22.67 23.5-32 7.99-19.75 27-29.5-27.65-23.7-15.5-58.5 7.33-16.82 20.5-29.5 10.79-8.14 22-15.5-16.49-37.08-5.5-76 3.19-6.13 7.5-11.5 1.48-0.89 2 1-5.69 41.09 12.5 78.5 1 1 2 2 9.97-3.24 20.5-4 2 0 4 0 0-7.5 0-15 0.99-42.22 24.5-77 6.12-7.12 14-12-4.65 13.43-10 27-11.93 37.6-9.5 77 49.38 0.7 83.5 36 2.75 4.5 5.5 9 38.99-52.24 93-88.5 45.84-29.03 100-32.5 15.69-1.56 29 6.5 5.68 7.29 3.5 16.5-10.38 33.62-43.5 45-4.39 37.33-41 45-0.79 8.63-6 15.5 1.91 1.83 4.5 2.5 22.27-17.25 50.5-14.5 12.93-9.41 28-15 36.22-8.28 31.5 28.5-15.19 51.69-62.5 77.5-65.92 35.87-138 15.5-19.67 10.42-42 10.5-8.39 2.88-17 5 3.58 6.08 10 9 20.92-1.14 36 13 22.67 5.23 34.5 25.5 3.33 7.13-3.5 11.5-3.88 1.8-8 3 7.36 8.45 6.5 19.5-4.43 5.66-11.5 3.5-12.84-5.67-26-10.5-39.4 21.02-83 10.5-18.85-5.78-36.5-14.5-13.65 4.14-23.5 14.5-9.51 3.74-11-6.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s2" d="m153.5 173.5q24.62 1.46 46 13.5 12.11 8.1 17.5 21.5 0.74 2.45 0.5 5 0.09 0.81 1 1 1.48-4.9 1-10 5.04 10.48 1.5 22-9.81 27.86-35.5 42.5-26.17 14.97-56 19.5-2.77-0.4-2 1 2.86 1.27 6 1 25.64 1.53 48.5-10 0.34 10.08 2 20 1.08 5.76 5 10 1 1.5 0 3-31.11 20.84-68.5 17.5-23.7-5.7-32.5-28.5-4.39-9.18-3.5-19 15.41 6.23 32 4.5-20.68-6.39-39-18-34.81-27.22-12.5-65.5 11.84-14.83 29-23 4.21 7.66 11.5 12.5 3 1 6 0-26.04-34.62-29-78-0.13-8.46 2-16.5 1 6.5 2 13 3.43 39.53 24.5 73 2.03 2.28 4.5 4 0.5-1.25 1-2.5-1.27-6.54-5-12 0.5-0.75 1-1.5 9.72-3.43 20-4 0.55 10.34 8 17.5 1.94 0.74 4 0.5-17.8-64.6 16.5-122 0.98-1.79 1.5 0-28.21 56.64-13.5 118 1.08 1.43 2.5 0.5 2.21-4.98 2-10.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s3" d="m454.5 97.5q-18.37-2.97-37-1.5-16.14 2.08-32 5.5 32.38-14.09 67-7.5 1.98 1.22 2 3.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s4" d="m454.5 97.5q-1.33 11.18-8.5 20-21.81 26.28-55.5 32-1.11-0.2-2 0.5 2.31 2.82 5.5 4.5 1 2 0 4-9.56 11.3-19.5 20 19.71-8.72 31-27 2.68-0.43 5 1-14.24 30.97-48 36.5-9.93 1.71-20 1.5-6.8-0.48-13 1 5.81 6.92 14 11-10.78 16.03-27 26.5 27.16-7.4 38-33.5 4.34 1.35 9 1-9.08 23.84-33 33.5-18.45 6.41-38 7 22.59 8.92 45-1 12.05-5.52 24-11 9.01-1.79 17 2.5 5.28-4.38 11-8 12.8-6.07 27-5 0 0.5 0 1-19.34 2.69-34 15.5 0.5 0.25 1 0.5 17.79-8.09 36-15 2.71-0.79 5-2 2.5-1 5-2 5.53-4.04 11-8 11.7-4.18 24-6.5 7.78-1.36 15 1.5-2.97 18.45-13.5 34-34.92 49.37-94.5 62.5-59.27 12.45-108-23-15.53-12.52-21.5-31.5-2.47-14.26 4-27-3.15 24.41 14 42-4.92-10.28-7-22-1.97-17.63 7-33 47.28-69.5 125.5-100 15.86-3.42 32-5.5 18.63-1.47 37 1.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s5" d="m86.5 112.5q-1-6.5-2-13 0.7-5.34 3.5-10-1.8 11.32-1.5 23z"/>
</g>
<g>
<path fill-rule="evenodd" class="s6" d="m433.5 97.5q2.22-0.39 4 1-10 13.75-27 14-0.24-2.06 0.5-4 10.3-7.78 22.5-11z"/>
</g>
<g>
<path fill-rule="evenodd" class="s7" d="m407.5 101.5q2.55-0.24 5 0.5-52.87 18.31-84.5 64.5-6.94 7.95-17 11-9.38-2.38-5-11 40.38-48.62 101.5-65z"/>
</g>
<g>
<path fill-rule="evenodd" class="s8" d="m402.5 112.5q3 0 6 0-2.56 8.8-12 7-0.22-1.58 0.5-3 2.72-2.22 5.5-4z"/>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
<path fill-rule="evenodd" class="s9" d="m390.5 149.5q7.77 0.52 15 2-11.29 18.28-31 27 9.94-8.7 19.5-20 1-2 0-4-3.19-1.68-5.5-4.5 0.89-0.7 2-0.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s10" d="m131.5 145.5q0 7.5 0 15-2 0-4 0 1.06-1.36 3-1-0.48-7.29 1-14z"/>
</g>
<g>
<path fill-rule="evenodd" class="s11" d="m219.5 204.5q-1 4.5-2 9 0.24-2.55-0.5-5-5.39-13.4-17.5-21.5-21.38-12.04-46-13.5 0-2 0-4 36.7-0.86 61.5 26 3.06 4.11 4.5 9z"/>
</g>
<g>
<path fill-rule="evenodd" class="s12" d="m329.5 191.5q6.2-1.48 13-1-3.5 1-7 2-2.9-0.97-6-1z"/>
</g>
<g>
<path fill-rule="evenodd" class="s13" d="m329.5 191.5q3.1 0.03 6 1 9.55 1.31 19 3-10.84 26.1-38 33.5 16.22-10.47 27-26.5-8.19-4.08-14-11z"/>
</g>
<g>
<path fill-rule="evenodd" class="s14" d="m479.5 199.5q-7.22-2.86-15-1.5-12.3 2.32-24 6.5 15.6-13.11 36-11.5 3.63 2.26 3 6.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s15" d="m193.5 216.5q-12.01 1.52-22 8-2.83 1.29-5.5 3-4.79-4.57-6.5-11-5.04 2.2-9.5-1-3.47-6.4 3.5-3 4.4 0.05 8-2.5 9.22-9.73 21-16 6.3-3.24 12 1-2.9 1.22-6 1.5 2.61 5.74 4.5 12 0.75 3.97 0.5 8z"/>
</g>
<g>
<path fill-rule="evenodd" class="s16" d="m458.5 200.5q3.04-0.24 6 0.5-18.02 7.05-33 19-1 1-2 0 11.53-14.3 29-19.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s17" d="m178.5 202.5q6.85-0.63 4.5 6-7.6 5.09-6-4 1.08-0.82 1.5-2z"/>
</g>
<g>
<path fill-rule="evenodd" class="s18" d="m469.5 201.5q-2.26 13.65-14.5 22-0.47-2.11 1-4 7.08-8.82 13.5-18z"/>
</g>
<g>
<path fill-rule="evenodd" class="s19" d="m74.5 208.5q8.22-0.2 16 2.5 11.8 4.26 23.5 8.5 5.65-0.63 8-6 2.41 11.83-9.5 13 0.55 3.61 2 7-0.5 1-1 2-4.67-0.94-9.5-1-9.96 0.44-19.5 2.5-5.05-3.55-6.5-9.5-0.75-7.48-0.5-15-6.47 0.15-3-4z"/>
</g>
<g>
<path fill-rule="evenodd" class="s20" d="m429.5 212.5q-2.5 1-5 2-4 0-8 0-14.2-1.07-27 5 15.27-12.44 35-9.5 2.72 1.14 5 2.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s21" d="m219.5 204.5q0.48 5.1-1 10-0.91-0.19-1-1 1-4.5 2-9z"/>
</g>
<g>
<path fill-rule="evenodd" class="s22" d="m416.5 215.5q0-0.5 0-1 4 0 8 0-2.29 1.21-5 2-1.06-1.36-3-1z"/>
</g>
<g>
<path fill-rule="evenodd" class="s23" d="m416.5 215.5q1.94-0.36 3 1-18.21 6.91-36 15-0.5-0.25-1-0.5 14.66-12.81 34-15.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s24" d="m193.5 216.5q4.39 1.3 9 3-0.79 1.04-2 1.5-14.77-0.13-29 3.5 9.99-6.48 22-8z"/>
</g>
<g>
<path fill-rule="evenodd" class="s25" d="m98.5 219.5q6.09-0.98 6 5-3.04 0.24-6-0.5-1.84-2.24 0-4.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s26" d="m176.5 229.5q8.85-1.14 16 4-4.98 1.75-10 0-13.56 14.3-33 19.5-28.06 8.2-55 1 3.32-6.4 10-5.5-0.71 1.47-2 2.5 36.58 4.24 69-14 4.68-2.13 1-5 2.35-0.91 4-2.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s27" d="m231.5 238.5q1.31-0.2 2 1-3.13 28.62 15 51-16.25 6.75-27-7.5-1-1-2 0 14.73 29.34 46 18.5 1.79 0.52 0 1.5-37.63 16.82-50.5-22.5-5.1-26.48 16.5-42z"/>
</g>
<g>
<path fill-rule="evenodd" class="s28" d="m243.5 259.5q5.88 3.62 10.5 9 12.96 18.46 32.5 29.5-31.51-7.75-43-38.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s29" d="m203.5 266.5q1.31-0.2 2 1-2.48 22.08 12 39-6.99 1.35-14 0.5 4.59 4.08 10 7-8.71 0.28-14.5-6.5-16.98-22.76 4.5-41z"/>
</g>
<g>
<path fill-rule="evenodd" class="s27" d="m58.5 284.5q9.6-2.17 14.5 6 5.15 14.18-1 28-11.05-13.14-27.5-17.5 5.15-9.9 14-16.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s30" d="m129.5 288.5q2 1 4 2-3.14 0.27-6-1-0.77-1.4 2-1z"/>
</g>
<g>
<path fill-rule="evenodd" class="s31" d="m56.5 313.5q3.43 5.43 8 10-4.88 0.44-8 4-1.11-0.2-2 0.5 28.91 1.65 38 28.5 0.45 3.16-1 6-11.02-7.01-23-12.5-4.75-3.75-9.5-7.5 1.47 7.42 7 13 8.34 27.18 32 43 0.99 2.41-1.5 3.5-40.25 5.58-66.5-25.5-15.67-22.01-8-48 10.46-23.87 34.5-15z"/>
</g>
<g>
<path fill-rule="evenodd" class="s32" d="m45.5 317.5q4.03-0.25 8 0.5 2.46 4.16-2 6-6.04 2.01-9-3.5 1.26-1.85 3-3z"/>
</g>
<g>
<path fill-rule="evenodd" class="s33" d="m56.5 313.5q4.91 3.14 9.5 7 0.88 2.25-1.5 3-4.57-4.57-8-10z"/>
</g>
<g>
<path fill-rule="evenodd" class="s34" d="m198.5 319.5q-11.1 11.56-27 15.5-15.75 4.88-32 2.5 28.81-3.69 54-18.5 2.65-0.96 5 0.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s4" d="m198.5 319.5q1.44 0.68 2.5 2 2.41 8.23 6 16 1.2 2.64-0.5 5-30.65 21.41-68 18.5-25.16-6.17-32.5-30.5 6.96 4.99 15.5 6.5 8.99 0.75 18 0.5 16.25 2.38 32-2.5 15.9-3.94 27-15.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s35" d="m92.5 356.5q-9.09-26.85-38-28.5 0.89-0.7 2-0.5 25.47-4.89 35.5 19 0.75 4.98 0.5 10z"/>
</g>
<g>
<path fill-rule="evenodd" class="s36" d="m72.5 335.5q3.62-0.38 5 3-4.22 1.83-5-3z"/>
</g>
<g>
<path fill-rule="evenodd" class="s37" d="m223.5 336.5q5.59-0.48 11 1-4.04 4.16-8.5 8-5.99-3.8-2.5-9z"/>
</g>
<g>
<path fill-rule="evenodd" class="s38" d="m90.5 334.5q0.59-1.54 2-0.5 3.94 5.45 9 10 7 6 14 12-6.91-1.7-13-6-6.21-7.72-12-15.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s39" d="m261.5 346.5q-3.54-2.44-8-3.5-6.98-0.75-14-0.5 0.63-1.08 2-1.5 13.82-2.52 26 4-2.63 1.98-6 1.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s40" d="m239.5 342.5q7.02-0.25 14 0.5 4.46 1.06 8 3.5-5.2 2.35-10 5.5-3.88 4.65-9 7.5-9.89-3.09-9.5-13 2.36-3.63 6.5-4z"/>
</g>
<g>
<path fill-rule="evenodd" class="s41" d="m214.5 349.5q-21.43 15.48-48 16 22.82-5.9 43-18.5 3.64-1.12 5 2.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s42" d="m214.5 349.5q5.96 7.2 13.5 13 1 1 0 2-28.58 23.34-65.5 20.5-18.15-4.24-27.5-19.5 1.13 0.94 2.5 1.5 14.7 1.42 29-1.5 26.57-0.52 48-16z"/>
</g>
<g>
<path fill-rule="evenodd" class="s43" d="m302.5 373.5q-14.74-16.73-37-19-4.55 0.25-9 1 25.3-10.24 43.5 11 2.85 2.91 2.5 7z"/>
</g>
<g>
<path fill-rule="evenodd" class="s44" d="m302.5 373.5q0.21 2.44-2 3.5-28.69 7.6-50.5-12.5-0.06-6.71 6.5-9 4.45-0.75 9-1 22.26 2.27 37 19z"/>
</g>
<g>
<path fill-rule="evenodd" class="s45" d="m100.5 356.5q5.42 2.71 11 5.5-13.04 7.54-18.5 21.5-7.57-7.14-10.5-17 5.58 1.54 10 5.5 4.2 0.84 5.5-3.5 1.41-5.99 2.5-12z"/>
</g>
<g>
<path fill-rule="evenodd" class="s8" d="m83.5 394.5q-18.9-10.15-29.5-29-1.54-3.52-2-7 5.79 2.39 10 7 7.82 16.63 21.5 29z"/>
</g>
<g>
<path fill-rule="evenodd" class="s46" d="m232.5 365.5q17.6 6.19 10.5 23-10.6 10.42-25.5 11.5-25.94 3.21-49-9 36.75-1.65 64-25.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s47" d="m113.5 367.5q7.7-0.01 9.5 7-9.69 7.19-18.5 15.5-7.23 5.76-5.5-3.5 3.12-12.84 14.5-19z"/>
</g>
<g>
<path fill-rule="evenodd" class="s29" d="m126.5 380.5q7.88-0.4 12 6.5-8.5 7.25-17 14.5-5.62-12.55 5-21z"/>
</g>
<g>
<path fill-rule="evenodd" class="s48" d="m283.5 385.5q3.22 2.95 7 5.5 2.8 4.03 6 7.5 0.42 2.77-2 4-15.5-9.75-31-19.5-1.79-0.98 0-1.5 9.96 2.49 20 4z"/>
</g>
<g>
<path fill-rule="evenodd" class="s49" d="m283.5 385.5q8.71-1.27 11.5 7 1.22 2.9 1.5 6-3.2-3.47-6-7.5-3.78-2.55-7-5.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s50" d="m83.5 394.5q1.88-0.06 3 1.5-2.25 0.88-3-1.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s51" d="m258.5 392.5q3.51 0.41 0 2.5-2.33 1.93-5 2 2.61-2.28 5-4.5z"/>
</g>
<g>
<path fill-rule="evenodd" class="s52" d="m111.5 392.5q0.09-0.81 1-1 1.48 4.9 1 10-1-4.5-2-9z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 13 KiB

@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" width="512" height="512"><svg width="512" height="512" viewBox="0 0 512 512" fill="none" xmlns="http://www.w3.org/2000/svg">
<rect width="512" height="512" fill="#131010"></rect>
<path d="M320 224V352H192V224H320Z" fill="#5A5858"></path>
<path fill-rule="evenodd" clip-rule="evenodd" d="M384 416H128V96H384V416ZM320 160H192V352H320V160Z" fill="white"></path>
</svg><style>@media (prefers-color-scheme: light) { :root { filter: none; } }
@media (prefers-color-scheme: dark) { :root { filter: none; } }
</style></svg>

After

Width:  |  Height:  |  Size: 612 B

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 800">
<rect width="800" height="800" rx="160" fill="#fff"/>
<path fill="#000" fill-rule="evenodd" d="
M165.29 165.29 H517.36 V400 H400 V517.36 H282.65 V634.72 H165.29 Z
M282.65 282.65 V400 H400 V282.65 Z
"/>
<path fill="#000" d="M517.36 400 H634.72 V634.72 H517.36 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 389 B

+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 800">
<rect width="800" height="800" rx="160" fill="#000"/>
<path fill="#fff" fill-rule="evenodd" d="
M165.29 165.29 H517.36 V400 H400 V517.36 H282.65 V634.72 H165.29 Z
M282.65 282.65 V400 H400 V282.65 Z
"/>
<path fill="#fff" d="M517.36 400 H634.72 V634.72 H517.36 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 389 B

+33 -8
View File
@@ -161,7 +161,7 @@ export async function getModels(query?: string): Promise<Model[]> {
// Add query if it's in the registry and not already in the list
if (!exactMatch) {
const result = await getModelUpstreamInfo(new Model({ model: query }));
const existsUpstream = !!result.digest && !result.error;
const existsUpstream = result.exists;
if (existsUpstream) {
filteredModels.push(new Model({ model: query }));
}
@@ -339,7 +339,7 @@ export async function deleteChat(chatId: string): Promise<void> {
// Get upstream information for model staleness checking
export async function getModelUpstreamInfo(
model: Model,
): Promise<{ digest?: string; pushTime: number; error?: string }> {
): Promise<{ stale: boolean; exists: boolean; error?: string }> {
try {
const response = await fetch(`${API_BASE}/api/v1/model/upstream`, {
method: "POST",
@@ -353,22 +353,22 @@ export async function getModelUpstreamInfo(
if (!response.ok) {
console.warn(
`Failed to check upstream digest for ${model.model}: ${response.status}`,
`Failed to check upstream for ${model.model}: ${response.status}`,
);
return { pushTime: 0 };
return { stale: false, exists: false };
}
const data = await response.json();
if (data.error) {
console.warn(`Upstream digest check: ${data.error}`);
return { error: data.error, pushTime: 0 };
console.warn(`Upstream check: ${data.error}`);
return { stale: false, exists: false, error: data.error };
}
return { digest: data.digest, pushTime: data.pushTime || 0 };
return { stale: !!data.stale, exists: true };
} catch (error) {
console.warn(`Error checking model staleness:`, error);
return { pushTime: 0 };
return { stale: false, exists: false };
}
}
@@ -406,6 +406,31 @@ export async function* pullModel(
}
}
export interface ModelRecommendation {
model: string;
description: string;
context_length?: number;
max_output_tokens?: number;
vram_bytes?: number;
}
export interface ModelRecommendationsResponse {
recommendations: ModelRecommendation[];
}
export async function getModelRecommendations(): Promise<ModelRecommendation[]> {
const response = await fetch(
`${API_BASE}/api/experimental/model-recommendations`,
);
if (!response.ok) {
throw new Error(
`Failed to fetch model recommendations: ${response.statusText}`,
);
}
const data: ModelRecommendationsResponse = await response.json();
return data.recommendations || [];
}
export async function getInferenceCompute(): Promise<InferenceComputeResponse> {
const response = await fetch(`${API_BASE}/api/v1/inference-compute`);
if (!response.ok) {
+28 -11
View File
@@ -480,13 +480,15 @@ function ChatForm({
return;
}
// Prepare attachments for submission
const attachmentsToSend: FileAttachment[] = message.attachments.map(
(att) => ({
// Prepare attachments for submission, excluding unsupported images
const attachmentsToSend: FileAttachment[] = message.attachments
.filter(
(att) => hasVisionCapability || !isImageFile(att.filename),
)
.map((att) => ({
filename: att.filename,
data: att.data || new Uint8Array(0), // Empty data for existing files
}),
);
}));
const useWebSearch =
supportsWebSearch && webSearchEnabled && !cloudDisabled;
@@ -736,10 +738,17 @@ function ChatForm({
)}
{(message.attachments.length > 0 || message.fileErrors.length > 0) && (
<div className="flex gap-2 overflow-x-auto px-3 pt pb-3 w-full scrollbar-hide">
{message.attachments.map((attachment, index) => (
{message.attachments.map((attachment, index) => {
const isUnsupportedImage =
!hasVisionCapability && isImageFile(attachment.filename);
return (
<div
key={attachment.id}
className="group flex items-center gap-2 py-2 px-3 rounded-lg bg-neutral-50 dark:bg-neutral-700/50 hover:bg-neutral-100 dark:hover:bg-neutral-700 transition-colors flex-shrink-0"
className={`group flex items-center gap-2 py-2 px-3 rounded-lg transition-colors flex-shrink-0 ${
isUnsupportedImage
? "bg-red-50 dark:bg-red-900/20 border border-red-200 dark:border-red-800"
: "bg-neutral-50 dark:bg-neutral-700/50 hover:bg-neutral-100 dark:hover:bg-neutral-700"
}`}
>
{isImageFile(attachment.filename) ? (
<ImageThumbnail
@@ -764,9 +773,16 @@ function ChatForm({
/>
</svg>
)}
<span className="text-sm text-neutral-700 dark:text-neutral-300 max-w-[150px] truncate">
{attachment.filename}
</span>
<div className="flex flex-col min-w-0">
<span className={`text-sm max-w-36 truncate ${isUnsupportedImage ? "text-red-700 dark:text-red-300" : "text-neutral-700 dark:text-neutral-300"}`}>
{attachment.filename}
</span>
{isUnsupportedImage && (
<span className="text-xs text-red-600 dark:text-red-400 opacity-75">
This model does not support images
</span>
)}
</div>
<button
type="button"
onClick={() => removeFile(index)}
@@ -788,7 +804,8 @@ function ChatForm({
</svg>
</button>
</div>
))}
);
})}
{message.fileErrors.map((fileError, index) => (
<div
key={`error-${index}`}
+27 -11
View File
@@ -6,12 +6,13 @@ import { getChat } from "@/api";
import { Link } from "@/components/ui/link";
import { useState, useRef, useEffect, useCallback, useMemo } from "react";
import { ChatsResponse } from "@/gotypes";
import { CogIcon } from "@heroicons/react/24/outline";
import { CogIcon, RocketLaunchIcon } from "@heroicons/react/24/outline";
// there's a hidden debug feature to copy a chat's data to the clipboard by
// holding shift and clicking this many times within this many seconds
const DEBUG_SHIFT_CLICKS_REQUIRED = 5;
const DEBUG_SHIFT_CLICK_WINDOW_MS = 7000; // 7 seconds
const launchSidebarRequestedKey = "ollama.launchSidebarRequested";
interface ChatSidebarProps {
currentChatId?: string;
@@ -267,9 +268,8 @@ export function ChatSidebar({ currentChatId }: ChatSidebarProps) {
<Link
href="/c/new"
mask={{ to: "/" }}
className={`flex w-full items-center gap-3 rounded-lg px-2 py-2 text-left text-sm text-neutral-700 hover:bg-neutral-100 dark:hover:bg-neutral-800 dark:text-neutral-100 ${
currentChatId === "new" ? "bg-neutral-100 dark:bg-neutral-800" : ""
}`}
className={`flex w-full items-center gap-3 rounded-lg px-2 py-2 text-left text-sm text-neutral-700 hover:bg-neutral-100 dark:hover:bg-neutral-800 dark:text-neutral-100 ${currentChatId === "new" ? "bg-neutral-100 dark:bg-neutral-800" : ""
}`}
draggable={false}
>
<svg
@@ -283,6 +283,23 @@ export function ChatSidebar({ currentChatId }: ChatSidebarProps) {
</svg>
<span className="truncate">New Chat</span>
</Link>
<Link
to="/c/$chatId"
params={{ chatId: "launch" }}
onClick={() => {
if (currentChatId !== "launch") {
sessionStorage.setItem(launchSidebarRequestedKey, "1");
}
}}
className={`flex w-full items-center gap-3 rounded-lg px-2 py-2 text-left text-sm text-neutral-700 hover:bg-neutral-100 dark:hover:bg-neutral-800 dark:text-neutral-100 cursor-pointer ${currentChatId === "launch"
? "bg-neutral-100 dark:bg-neutral-800"
: ""
}`}
draggable={false}
>
<RocketLaunchIcon className="h-5 w-5 stroke-current" />
<span className="truncate">Launch</span>
</Link>
{isWindows && (
<Link
href="/settings"
@@ -304,19 +321,18 @@ export function ChatSidebar({ currentChatId }: ChatSidebarProps) {
{group.chats.map((chat) => (
<div
key={chat.id}
className={`allow-context-menu flex items-center relative text-sm text-neutral-800 dark:text-neutral-400 rounded-lg hover:bg-neutral-100 dark:hover:bg-neutral-800 ${
chat.id === currentChatId
? "bg-neutral-100 text-black dark:bg-neutral-800"
: ""
}`}
className={`allow-context-menu flex items-center relative text-sm text-neutral-800 dark:text-neutral-400 rounded-lg hover:bg-neutral-100 dark:hover:bg-neutral-800 ${chat.id === currentChatId
? "bg-neutral-100 text-black dark:bg-neutral-800"
: ""
}`}
onMouseEnter={() => handleMouseEnter(chat.id)}
onContextMenu={(e) =>
handleContextMenu(
e,
chat.id,
chat.title ||
chat.userExcerpt ||
chat.createdAt.toLocaleString(),
chat.userExcerpt ||
chat.createdAt.toLocaleString(),
)
}
>
+4
View File
@@ -10,6 +10,7 @@ interface CopyButtonProps {
showLabels?: boolean;
className?: string;
title?: string;
onCopy?: () => void;
}
const CopyButton: React.FC<CopyButtonProps> = ({
@@ -20,6 +21,7 @@ const CopyButton: React.FC<CopyButtonProps> = ({
showLabels = false,
className = "",
title = "",
onCopy,
}) => {
const [isCopied, setIsCopied] = useState(false);
@@ -48,12 +50,14 @@ const CopyButton: React.FC<CopyButtonProps> = ({
}
setIsCopied(true);
onCopy?.();
setTimeout(() => setIsCopied(false), 2000);
} catch (error) {
console.error("Clipboard API failed, falling back to plain text", error);
try {
await navigator.clipboard.writeText(content);
setIsCopied(true);
onCopy?.();
setTimeout(() => setIsCopied(false), 2000);
} catch (fallbackError) {
console.error("Fallback copy also failed:", fallbackError);
@@ -0,0 +1,158 @@
import { useSettings } from "@/hooks/useSettings";
import CopyButton from "@/components/CopyButton";
interface LaunchCommand {
id: string;
name: string;
command: string;
description: string;
icon: string;
darkIcon?: string;
iconClassName?: string;
borderless?: boolean;
}
const LAUNCH_COMMANDS: LaunchCommand[] = [
{
id: "claude-desktop",
name: "Claude Desktop",
command: "ollama launch claude-desktop",
description: "Claude Desktop with Ollama Cloud",
icon: "/launch-icons/claude.svg",
iconClassName: "h-7 w-7",
},
{
id: "claude",
name: "Claude Code",
command: "ollama launch claude",
description: "Anthropic's coding tool with subagents",
icon: "/launch-icons/claude-code.svg",
iconClassName: "h-7 w-7",
},
{
id: "openclaw",
name: "OpenClaw",
command: "ollama launch openclaw",
description: "Personal AI with 100+ skills",
icon: "/launch-icons/openclaw.svg",
},
{
id: "hermes",
name: "Hermes Agent",
command: "ollama launch hermes",
description: "Self-improving AI agent built by Nous Research",
icon: "/launch-icons/hermes-agent.svg",
iconClassName: "h-7 w-7",
},
{
id: "opencode",
name: "OpenCode",
command: "ollama launch opencode",
description: "Anomaly's open-source coding agent",
icon: "/launch-icons/opencode.svg",
iconClassName: "h-7 w-7 rounded",
},
{
id: "codex",
name: "Codex",
command: "ollama launch codex",
description: "OpenAI's open-source coding agent",
icon: "/launch-icons/codex.svg",
darkIcon: "/launch-icons/codex-dark.svg",
iconClassName: "h-7 w-7",
},
{
id: "copilot",
name: "Copilot CLI",
command: "ollama launch copilot",
description: "GitHub's AI coding agent for the terminal",
icon: "/launch-icons/copilot.svg",
darkIcon: "/launch-icons/copilot-dark.svg",
iconClassName: "h-7 w-7",
},
{
id: "droid",
name: "Droid",
command: "ollama launch droid",
description: "Factory's coding agent across terminal and IDEs",
icon: "/launch-icons/droid.svg",
},
{
id: "pi",
name: "Pi",
command: "ollama launch pi",
description: "Minimal AI agent toolkit with plugin support",
icon: "/launch-icons/pi.svg",
darkIcon: "/launch-icons/pi-dark.svg",
iconClassName: "h-7 w-7",
},
];
export default function LaunchCommands() {
const isWindows = navigator.platform.toLowerCase().includes("win");
const { setSettings } = useSettings();
const renderCommandCard = (item: LaunchCommand) => (
<div key={item.command} className="w-full text-left">
<div className="flex items-start gap-4 sm:gap-5">
<div
aria-hidden="true"
className={`flex h-10 w-10 shrink-0 items-center justify-center rounded-lg overflow-hidden ${item.borderless ? "" : "border border-neutral-200 bg-white dark:border-neutral-700 dark:bg-neutral-900"}`}
>
{item.darkIcon ? (
<picture>
<source srcSet={item.darkIcon} media="(prefers-color-scheme: dark)" />
<img src={item.icon} alt="" className={`${item.iconClassName ?? "h-8 w-8"} rounded-sm`} />
</picture>
) : (
<img src={item.icon} alt="" className={item.borderless ? "h-full w-full rounded-xl" : `${item.iconClassName ?? "h-8 w-8"} rounded-sm`} />
)}
</div>
<div className="min-w-0 flex-1">
<span className="text-sm font-medium text-neutral-900 dark:text-neutral-100">
{item.name}
</span>
<p className="mt-0.5 text-xs text-neutral-500 dark:text-neutral-400">
{item.description}
</p>
<div className="mt-2 flex items-center gap-2 rounded-xl border-neutral-200 dark:border-neutral-700 bg-neutral-50 dark:bg-neutral-800 px-3 py-2">
<code className="min-w-0 flex-1 truncate text-xs text-neutral-600 dark:text-neutral-300">
{item.command}
</code>
<CopyButton
content={item.command}
size="md"
title="Copy command to clipboard"
className="text-neutral-500 dark:text-neutral-400 hover:text-neutral-700 dark:hover:text-neutral-200 hover:bg-neutral-200/60 dark:hover:bg-neutral-700/70"
onCopy={() => {
setSettings({ LastHomeView: item.id }).catch(() => { });
}}
/>
</div>
</div>
</div>
</div>
);
return (
<main className="flex h-screen w-full flex-col relative">
<section
className={`flex-1 overflow-y-auto overscroll-contain relative min-h-0 ${isWindows ? "xl:pt-4" : "xl:pt-8"}`}
>
<div className="max-w-[730px] mx-auto w-full px-4 pt-4 pb-20 sm:px-6 sm:pt-6 sm:pb-24 lg:px-8 lg:pt-8 lg:pb-28">
<h1 className="text-xl font-semibold text-neutral-900 dark:text-neutral-100">
Launch
</h1>
<p className="mt-1 text-sm text-neutral-500 dark:text-neutral-400">
Copy a command and run it in your terminal.
</p>
<div className="mt-6 grid gap-7">
{LAUNCH_COMMANDS.map(renderCommandCard)}
</div>
</div>
</section>
</main>
);
}
+2 -2
View File
@@ -536,7 +536,7 @@ function ToolCallDisplay({
let args: Record<string, unknown> | null = null;
try {
args = JSON.parse(toolCall.function.arguments) as Record<string, unknown>;
} catch (e) {
} catch {
args = null;
}
const query = args && typeof args.query === "string" ? args.query : "";
@@ -562,7 +562,7 @@ function ToolCallDisplay({
let args: Record<string, unknown> | null = null;
try {
args = JSON.parse(toolCall.function.arguments) as Record<string, unknown>;
} catch (e) {
} catch {
args = null;
}
const url = args && typeof args.url === "string" ? args.url : "";
+1 -1
View File
@@ -73,7 +73,7 @@ export default function MessageList({
? String(args.url).trim()
: "";
if (candidate) lastQuery = candidate;
} catch {}
} catch { /* ignored */ }
}
}
}
+1 -18
View File
@@ -61,24 +61,7 @@ export const ModelPicker = forwardRef<
try {
const upstreamInfo = await getModelUpstreamInfo(model);
// Compare local digest with upstream digest
let isStale =
model.digest &&
upstreamInfo.digest &&
model.digest !== upstreamInfo.digest;
// If the model has a modified time and upstream has a push time,
// check if the model was modified after the push time - if so, it's not stale
if (isStale && model.modified_at && upstreamInfo.pushTime > 0) {
const modifiedAtTime =
new Date(model.modified_at as string | number | Date).getTime() /
1000;
if (modifiedAtTime > upstreamInfo.pushTime) {
isStale = false;
}
}
if (isStale) {
if (upstreamInfo.stale) {
const currentStaleModels =
queryClient.getQueryData<Map<string, boolean>>(["staleModels"]) ||
new Map();
+7 -2
View File
@@ -214,6 +214,7 @@ export default function Settings() {
Agent: false,
Tools: false,
ContextLength: 0,
AutoUpdateEnabled: true,
});
updateSettingsMutation.mutate(defaultSettings);
}
@@ -272,6 +273,10 @@ export default function Settings() {
}
const isWindows = navigator.platform.toLowerCase().includes("win");
const handleCloseSettings = () => {
const chatId = settings.LastHomeView === "chat" ? "new" : "launch";
navigate({ to: "/c/$chatId", params: { chatId } });
};
return (
<main className="flex h-screen w-full flex-col select-none dark:bg-neutral-900">
@@ -285,7 +290,7 @@ export default function Settings() {
>
{isWindows && (
<button
onClick={() => navigate({ to: "/" })}
onClick={handleCloseSettings}
className="hover:bg-neutral-100 mr-3 dark:hover:bg-neutral-800 rounded-full p-1.5"
>
<ArrowLeftIcon className="w-5 h-5 dark:text-white" />
@@ -295,7 +300,7 @@ export default function Settings() {
</h1>
{!isWindows && (
<button
onClick={() => navigate({ to: "/" })}
onClick={handleCloseSettings}
className="p-1 hover:bg-neutral-100 mr-3 dark:hover:bg-neutral-800 rounded-full"
>
<XMarkIcon className="w-6 h-6 dark:text-white" />
+1 -1
View File
@@ -65,7 +65,7 @@ export const BadgeButton = forwardRef(function BadgeButton(
),
ref: React.ForwardedRef<HTMLElement>,
) {
let classes = clsx(
const classes = clsx(
className,
"group relative inline-flex rounded-md focus:not-data-focus:outline-hidden data-focus:outline-2 data-focus:outline-offset-2 data-focus:outline-blue-500",
);
+1 -1
View File
@@ -171,7 +171,7 @@ export const Button = forwardRef(function Button(
{ color, outline, plain, className, children, ...props }: ButtonProps,
ref: React.ForwardedRef<HTMLElement>,
) {
let classes = clsx(
const classes = clsx(
className,
styles.base,
outline
+5 -5
View File
@@ -381,7 +381,7 @@ export const useSendMessage = (chatId: string) => {
role: "assistant",
content: "",
thinking: "",
model: effectiveModel,
model: effectiveModel.model,
}),
);
lastMessage = newMessages[newMessages.length - 1];
@@ -433,7 +433,7 @@ export const useSendMessage = (chatId: string) => {
role: "assistant",
content: "",
thinking: "",
model: effectiveModel,
model: effectiveModel.model,
}),
);
lastMessage = newMessages[newMessages.length - 1];
@@ -520,7 +520,7 @@ export const useSendMessage = (chatId: string) => {
thinkingTimeStart:
lastMessage.thinkingTimeStart || event.thinkingTimeStart,
thinkingTimeEnd: event.thinkingTimeEnd,
model: selectedModel,
model: selectedModel.model,
});
newMessages[newMessages.length - 1] = updatedMessage;
} else {
@@ -533,7 +533,7 @@ export const useSendMessage = (chatId: string) => {
tool_calls: event.toolCalls,
thinkingTimeStart: event.thinkingTimeStart,
thinkingTimeEnd: event.thinkingTimeEnd,
model: selectedModel,
model: selectedModel.model,
}),
);
}
@@ -699,7 +699,7 @@ export const useSendMessage = (chatId: string) => {
queryClient.setQueryData(["chat", newId], {
chat: new Chat({
id: newId,
model: effectiveModel,
model: effectiveModel.model,
messages: [
new Message({
role: "user",
+13
View File
@@ -0,0 +1,13 @@
import { useQuery } from "@tanstack/react-query";
import { getModelRecommendations } from "@/api";
import type { ModelRecommendation } from "@/api";
export function useFeaturedModels() {
return useQuery<ModelRecommendation[], Error>({
queryKey: ["modelRecommendations"],
queryFn: getModelRecommendations,
staleTime: 5 * 60 * 1000,
gcTime: 30 * 60 * 1000,
refetchOnWindowFocus: false,
});
}
+35 -24
View File
@@ -1,51 +1,49 @@
import { useQuery } from "@tanstack/react-query";
import { Model } from "@/gotypes";
import { getModels } from "@/api";
import { mergeModels } from "@/utils/mergeModels";
import { useMemo } from "react";
import { useCloudStatus } from "./useCloudStatus";
import { useFeaturedModels } from "./useFeaturedModels";
export function useModels(searchQuery = "") {
const { cloudDisabled } = useCloudStatus();
const { data: recommendations, isLoading: recommendationsLoading } =
useFeaturedModels();
const localQuery = useQuery<Model[], Error>({
queryKey: ["models", searchQuery],
queryFn: () => getModels(searchQuery),
gcTime: 10 * 60 * 1000, // Keep in cache for 10 minutes
gcTime: 10 * 60 * 1000,
retry: 10,
// exponential backoff, starting at 100ms and capping at 5s
retryDelay: (attemptIndex) => Math.min(100 * 2 ** attemptIndex, 5000),
refetchOnWindowFocus: true,
refetchInterval: 30 * 1000, // Refetch every 30 seconds to keep models updated
refetchInterval: 30 * 1000,
refetchIntervalInBackground: true,
});
const allModels = useMemo(() => {
const models = mergeModels(localQuery.data || [], cloudDisabled);
const local = localQuery.data || [];
const featured = (recommendations || []).map((r) => r.model);
const featuredSet = new Set(featured);
if (searchQuery && searchQuery.trim()) {
const query = searchQuery.toLowerCase().trim();
const filteredModels = models.filter((model) =>
model.model.toLowerCase().includes(query),
);
// Recommended models first (using the local copy when downloaded),
// then everything else from /api/tags in tags order.
const recommended = featured.map(
(name) =>
local.find((m) => m.model === name) || new Model({ model: name }),
);
const rest = local.filter((m) => !featuredSet.has(m.model));
const merged = [...recommended, ...rest];
const seen = new Set<string>();
return filteredModels.filter((model) => {
const currentModel = model.model.toLowerCase();
if (seen.has(currentModel)) {
return false;
}
seen.add(currentModel);
return true;
});
}
return models;
}, [localQuery.data, searchQuery, cloudDisabled]);
const visible = cloudDisabled
? merged.filter((m) => !m.isCloud())
: merged;
return filterBySearch(visible, searchQuery);
}, [localQuery.data, searchQuery, cloudDisabled, recommendations]);
return {
...localQuery,
data: allModels,
isLoading: localQuery.isLoading,
isLoading: localQuery.isLoading || recommendationsLoading,
};
}
@@ -53,3 +51,16 @@ export function useRefetchModels() {
const { refetch } = useModels();
return refetch;
}
function filterBySearch(models: Model[], query: string): Model[] {
const q = query.trim().toLowerCase();
if (!q) return models;
const seen = new Set<string>();
return models.filter((m) => {
const name = m.model.toLowerCase();
if (!name.includes(q) || seen.has(name)) return false;
seen.add(name);
return true;
});
}
+1 -4
View File
@@ -4,7 +4,6 @@ import { useModels } from "./useModels";
import { useChat } from "./useChats";
import { useSettings } from "./useSettings.ts";
import { Model } from "@/gotypes";
import { FEATURED_MODELS } from "@/utils/mergeModels";
import { getTotalVRAM } from "@/utils/vram.ts";
import { getInferenceCompute } from "@/api";
import { useCloudStatus } from "./useCloudStatus";
@@ -92,9 +91,7 @@ export function useSelectedModel(currentChatId?: string, searchQuery?: string) {
(settings.selectedModel &&
new Model({
model: settings.selectedModel,
cloud: FEATURED_MODELS.some(
(f) => f.endsWith("cloud") && f === settings.selectedModel,
),
cloud: settings.selectedModel.endsWith("cloud"),
ollama_host: false,
})) ||
null
+3
View File
@@ -9,6 +9,7 @@ interface SettingsState {
webSearchEnabled: boolean;
selectedModel: string;
sidebarOpen: boolean;
lastHomeView: string;
thinkEnabled: boolean;
thinkLevel: string;
}
@@ -21,6 +22,7 @@ type SettingsUpdate = Partial<{
ThinkLevel: string;
SelectedModel: string;
SidebarOpen: boolean;
LastHomeView: string;
}>;
export function useSettings() {
@@ -50,6 +52,7 @@ export function useSettings() {
thinkLevel: settingsData?.settings?.ThinkLevel ?? "none",
selectedModel: settingsData?.settings?.SelectedModel ?? "",
sidebarOpen: settingsData?.settings?.SidebarOpen ?? false,
lastHomeView: settingsData?.settings?.LastHomeView ?? "launch",
}),
[settingsData?.settings],
);
+94 -4
View File
@@ -4,12 +4,37 @@ import Chat from "@/components/Chat";
import { getChat } from "@/api";
import { SidebarLayout } from "@/components/layout/layout";
import { ChatSidebar } from "@/components/ChatSidebar";
import LaunchCommands from "@/components/LaunchCommands";
import { useEffect, useRef } from "react";
import { useSettings } from "@/hooks/useSettings";
const launchSidebarRequestedKey = "ollama.launchSidebarRequested";
const launchSidebarSeenKey = "ollama.launchSidebarSeen";
const fallbackSessionState = new Map<string, string>();
function getSessionState() {
if (typeof sessionStorage !== "undefined") {
return sessionStorage;
}
return {
getItem(key: string) {
return fallbackSessionState.get(key) ?? null;
},
setItem(key: string, value: string) {
fallbackSessionState.set(key, value);
},
removeItem(key: string) {
fallbackSessionState.delete(key);
},
};
}
export const Route = createFileRoute("/c/$chatId")({
component: RouteComponent,
loader: async ({ context, params }) => {
// Skip loading for "new" chat
if (params.chatId !== "new") {
// Skip loading for special non-chat views
if (params.chatId !== "new" && params.chatId !== "launch") {
context.queryClient.ensureQueryData({
queryKey: ["chat", params.chatId],
queryFn: () => getChat(params.chatId),
@@ -21,13 +46,70 @@ export const Route = createFileRoute("/c/$chatId")({
function RouteComponent() {
const { chatId } = Route.useParams();
const { settingsData, setSettings } = useSettings();
const previousChatIdRef = useRef<string | null>(null);
// Always call hooks at the top level - use a flag to skip data when chatId is "new"
// Always call hooks at the top level - use a flag to skip data when chatId is a special view
const {
data: chatData,
isLoading: chatLoading,
error: chatError,
} = useChat(chatId === "new" ? "" : chatId);
} = useChat(chatId === "new" || chatId === "launch" ? "" : chatId);
useEffect(() => {
if (!settingsData) {
return;
}
const previousChatId = previousChatIdRef.current;
previousChatIdRef.current = chatId;
if (chatId === "launch") {
const sessionState = getSessionState();
const shouldOpenSidebar =
previousChatId !== "launch" &&
(() => {
if (sessionState.getItem(launchSidebarRequestedKey) === "1") {
sessionState.removeItem(launchSidebarRequestedKey);
sessionState.setItem(launchSidebarSeenKey, "1");
return true;
}
if (sessionState.getItem(launchSidebarSeenKey) !== "1") {
sessionState.setItem(launchSidebarSeenKey, "1");
return true;
}
return false;
})();
const updates: { LastHomeView?: string; SidebarOpen?: boolean } = {};
if (settingsData.LastHomeView !== "launch") {
updates.LastHomeView = "launch";
}
if (shouldOpenSidebar && !settingsData.SidebarOpen) {
updates.SidebarOpen = true;
}
if (Object.keys(updates).length === 0) {
return;
}
setSettings(updates).catch(() => {
// Best effort persistence for home view preference.
});
return;
}
if (settingsData.LastHomeView === "chat") {
return;
}
setSettings({ LastHomeView: "chat" }).catch(() => {
// Best effort persistence for home view preference.
});
}, [chatId, settingsData, setSettings]);
// Handle "new" chat case - just use Chat component which handles everything
if (chatId === "new") {
@@ -38,6 +120,14 @@ function RouteComponent() {
);
}
if (chatId === "launch") {
return (
<SidebarLayout sidebar={<ChatSidebar currentChatId={chatId} />}>
<LaunchCommands />
</SidebarLayout>
);
}
// Handle existing chat case
if (chatLoading) {
return (
+10 -2
View File
@@ -1,10 +1,18 @@
import { createFileRoute, redirect } from "@tanstack/react-router";
import { getSettings } from "@/api";
export const Route = createFileRoute("/")({
beforeLoad: () => {
beforeLoad: async ({ context }) => {
const settingsData = await context.queryClient.ensureQueryData({
queryKey: ["settings"],
queryFn: getSettings,
});
const chatId =
settingsData?.settings?.LastHomeView === "chat" ? "new" : "launch";
throw redirect({
to: "/c/$chatId",
params: { chatId: "new" },
params: { chatId },
mask: {
to: "/",
},
+57
View File
@@ -0,0 +1,57 @@
import { describe, expect, it, vi, beforeEach } from "vitest";
import { copyTextToClipboard } from "./clipboard";
describe("copyTextToClipboard", () => {
beforeEach(() => {
vi.restoreAllMocks();
});
it("copies via Clipboard API when available", async () => {
const writeText = vi.fn().mockResolvedValue(undefined);
vi.stubGlobal("navigator", {
clipboard: {
writeText,
},
});
const copied = await copyTextToClipboard("ollama launch claude");
expect(copied).toBe(true);
expect(writeText).toHaveBeenCalledWith("ollama launch claude");
});
it("falls back to execCommand when Clipboard API fails", async () => {
const writeText = vi.fn().mockRejectedValue(new Error("not allowed"));
vi.stubGlobal("navigator", {
clipboard: {
writeText,
},
});
const textarea = {
value: "",
setAttribute: vi.fn(),
style: {} as Record<string, string>,
focus: vi.fn(),
select: vi.fn(),
};
const appendChild = vi.fn();
const removeChild = vi.fn();
const execCommand = vi.fn().mockReturnValue(true);
vi.stubGlobal("document", {
createElement: vi.fn().mockReturnValue(textarea),
body: {
appendChild,
removeChild,
},
execCommand,
});
const copied = await copyTextToClipboard("ollama launch openclaw");
expect(copied).toBe(true);
expect(execCommand).toHaveBeenCalledWith("copy");
expect(appendChild).toHaveBeenCalled();
expect(removeChild).toHaveBeenCalled();
});
});
+30
View File
@@ -0,0 +1,30 @@
export async function copyTextToClipboard(text: string): Promise<boolean> {
try {
await navigator.clipboard.writeText(text);
return true;
} catch (clipboardError) {
console.error(
"Clipboard API failed, falling back to execCommand",
clipboardError,
);
}
try {
const textarea = document.createElement("textarea");
textarea.value = text;
textarea.setAttribute("readonly", "true");
textarea.style.position = "fixed";
textarea.style.left = "-9999px";
textarea.style.opacity = "0";
document.body.appendChild(textarea);
textarea.focus();
textarea.select();
const copied = document.execCommand("copy");
document.body.removeChild(textarea);
return copied;
} catch (fallbackError) {
console.error("Fallback copy failed", fallbackError);
return false;
}
}
+5 -3
View File
@@ -29,13 +29,15 @@ describe("fileValidation", () => {
expect(result.valid).toBe(true);
});
it("should reject WebP images when vision capability is disabled", () => {
it("should accept images regardless of vision capability", () => {
// Vision capability check is handled at the UI layer (ChatForm),
// not at validation time, so users can switch models without
// needing to re-upload files.
const file = createMockFile("test.webp", 1024, "image/webp");
const result = validateFile(file, {
hasVisionCapability: false,
});
expect(result.valid).toBe(false);
expect(result.error).toBe("This model does not support images");
expect(result.valid).toBe(true);
});
it("should accept PNG images when vision capability is enabled", () => {
-5
View File
@@ -63,7 +63,6 @@ export function validateFile(
const {
maxFileSize = 10,
allowedExtensions = [...TEXT_FILE_EXTENSIONS, ...IMAGE_EXTENSIONS],
hasVisionCapability = false,
customValidator,
} = options;
@@ -83,10 +82,6 @@ export function validateFile(
return { valid: false, error: "File type not supported" };
}
if (IMAGE_EXTENSIONS.includes(fileExtension) && !hasVisionCapability) {
return { valid: false, error: "This model does not support images" };
}
// File size validation
if (file.size > MAX_FILE_SIZE) {
return { valid: false, error: "File too large" };
-128
View File
@@ -1,128 +0,0 @@
import { describe, it, expect } from "vitest";
import { Model } from "@/gotypes";
import { mergeModels, FEATURED_MODELS } from "@/utils/mergeModels";
import "@/api";
describe("Model merging logic", () => {
it("should handle cloud models with -cloud suffix", () => {
const localModels: Model[] = [
new Model({ model: "gpt-oss:120b-cloud" }),
new Model({ model: "llama3:latest" }),
new Model({ model: "mistral:latest" }),
];
const merged = mergeModels(localModels);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m: string) =>
m.endsWith("cloud"),
);
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m: string) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Verify local models are preserved and come after featured models
const featuredCount = FEATURED_MODELS.length;
expect(merged[featuredCount].model).toBe("llama3:latest");
expect(merged[featuredCount + 1].model).toBe("mistral:latest");
// Length should be exactly featured models plus our local models
expect(merged.length).toBe(FEATURED_MODELS.length + 2);
});
it("should hide cloud models when cloud is disabled", () => {
const localModels: Model[] = [
new Model({ model: "gpt-oss:120b-cloud" }),
new Model({ model: "llama3:latest" }),
new Model({ model: "mistral:latest" }),
];
const merged = mergeModels(localModels, true); // cloud disabled = true
// No cloud models should be present
const cloudModels = merged.filter((m) => m.isCloud());
expect(cloudModels.length).toBe(0);
// Should have non-cloud featured models
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Local models should be preserved
const featuredCount = nonCloudFeatured.length;
expect(merged[featuredCount].model).toBe("llama3:latest");
expect(merged[featuredCount + 1].model).toBe("mistral:latest");
});
it("should handle empty input", () => {
const merged = mergeModels([]);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m) => m.endsWith("cloud"));
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Length should be exactly FEATURED_MODELS length
expect(merged.length).toBe(FEATURED_MODELS.length);
});
it("should sort models correctly", () => {
const localModels: Model[] = [
new Model({ model: "zephyr:latest" }),
new Model({ model: "alpha:latest" }),
new Model({ model: "gpt-oss:120b-cloud" }),
];
const merged = mergeModels(localModels);
// First verify cloud models are first and in FEATURED_MODELS order
const cloudModels = FEATURED_MODELS.filter((m) => m.endsWith("cloud"));
for (let i = 0; i < cloudModels.length; i++) {
expect(merged[i].model).toBe(cloudModels[i]);
expect(merged[i].isCloud()).toBe(true);
}
// Then verify non-cloud featured models are next and in FEATURED_MODELS order
const nonCloudFeatured = FEATURED_MODELS.filter(
(m) => !m.endsWith("cloud"),
);
for (let i = 0; i < nonCloudFeatured.length; i++) {
const model = merged[i + cloudModels.length];
expect(model.model).toBe(nonCloudFeatured[i]);
expect(model.isCloud()).toBe(false);
}
// Non-featured local models should be at the end in alphabetical order
const featuredCount = FEATURED_MODELS.length;
expect(merged[featuredCount].model).toBe("alpha:latest");
expect(merged[featuredCount + 1].model).toBe("zephyr:latest");
});
});
-101
View File
@@ -1,101 +0,0 @@
import { Model } from "@/gotypes";
// Featured models list (in priority order)
export const FEATURED_MODELS = [
"gpt-oss:120b-cloud",
"gpt-oss:20b-cloud",
"deepseek-v3.1:671b-cloud",
"qwen3-coder:480b-cloud",
"qwen3-vl:235b-cloud",
"minimax-m2:cloud",
"glm-4.6:cloud",
"gpt-oss:120b",
"gpt-oss:20b",
"gemma3:27b",
"gemma3:12b",
"gemma3:4b",
"gemma3:1b",
"deepseek-r1:8b",
"qwen3-coder:30b",
"qwen3-vl:30b",
"qwen3-vl:8b",
"qwen3-vl:4b",
"qwen3:30b",
"qwen3:8b",
"qwen3:4b",
];
function alphabeticalSort(a: Model, b: Model): number {
return a.model.toLowerCase().localeCompare(b.model.toLowerCase());
}
//Merges models, sorting cloud models first, then other models
export function mergeModels(
localModels: Model[],
hideCloudModels: boolean = false,
): Model[] {
const allModels = (localModels || []).map((model) => model);
// 1. Get cloud models from local models and featured list
const cloudModels = [...allModels.filter((m) => m.isCloud())];
// Add any cloud models from FEATURED_MODELS that aren't in local models
FEATURED_MODELS.filter((f) => f.endsWith("cloud")).forEach((cloudModel) => {
if (!cloudModels.some((m) => m.model === cloudModel)) {
cloudModels.push(new Model({ model: cloudModel }));
}
});
// 2. Get other featured models (non-cloud)
const featuredModels = FEATURED_MODELS.filter(
(f) => !f.endsWith("cloud"),
).map((model) => {
// Check if this model exists in local models
const localMatch = allModels.find(
(m) => m.model.toLowerCase() === model.toLowerCase(),
);
if (localMatch) return localMatch;
return new Model({
model,
});
});
// 3. Get remaining local models that aren't featured and aren't cloud models
const remainingModels = allModels.filter(
(model) =>
!model.isCloud() &&
!FEATURED_MODELS.some(
(f) => f.toLowerCase() === model.model.toLowerCase(),
),
);
cloudModels.sort((a, b) => {
const aIndex = FEATURED_MODELS.indexOf(a.model);
const bIndex = FEATURED_MODELS.indexOf(b.model);
// If both are featured, sort by their position in FEATURED_MODELS
if (aIndex !== -1 && bIndex !== -1) {
return aIndex - bIndex;
}
// If only one is featured, featured model comes first
if (aIndex !== -1 && bIndex === -1) return -1;
if (aIndex === -1 && bIndex !== -1) return 1;
// If neither is featured, sort alphabetically
return a.model.toLowerCase().localeCompare(b.model.toLowerCase());
});
featuredModels.sort(
(a, b) =>
FEATURED_MODELS.indexOf(a.model) - FEATURED_MODELS.indexOf(b.model),
);
remainingModels.sort(alphabeticalSort);
return hideCloudModels
? [...featuredModels, ...remainingModels]
: [...cloudModels, ...featuredModels, ...remainingModels];
}
+2 -3
View File
@@ -133,9 +133,8 @@ type Error struct {
}
type ModelUpstreamResponse struct {
Digest string `json:"digest,omitempty"`
PushTime int64 `json:"pushTime"`
Error string `json:"error,omitempty"`
Stale bool `json:"stale"`
Error string `json:"error,omitempty"`
}
// Serializable data for the browser state
+28 -12
View File
@@ -32,6 +32,7 @@ import (
"github.com/ollama/ollama/app/version"
ollamaAuth "github.com/ollama/ollama/auth"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/manifest"
"github.com/ollama/ollama/types/model"
_ "github.com/tkrajina/typescriptify-golang-structs/typescriptify"
)
@@ -155,7 +156,7 @@ func (s *Server) ollamaProxy() http.Handler {
return
}
target := envconfig.Host()
target := envconfig.ConnectableHost()
s.log().Info("configuring ollama proxy", "target", target.String())
newProxy := httputil.NewSingleHostReverseProxy(target)
@@ -193,7 +194,7 @@ func (s *Server) Handler() http.Handler {
if CORS() {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Requested-With")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization, User-Agent, Accept, X-Requested-With")
w.Header().Set("Access-Control-Allow-Credentials", "true")
// Handle preflight requests
@@ -301,6 +302,7 @@ func (s *Server) Handler() http.Handler {
mux.Handle("HEAD /api/version", ollamaProxy)
mux.Handle("POST /api/me", ollamaProxy)
mux.Handle("POST /api/signout", ollamaProxy)
mux.Handle("GET /api/experimental/model-recommendations", ollamaProxy)
// React app - catch all non-API routes and serve the React app
mux.Handle("GET /", s.appHandler())
@@ -318,7 +320,7 @@ func (s *Server) handleError(w http.ResponseWriter, e error) {
if CORS() {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Requested-With")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization, User-Agent, Accept, X-Requested-With")
w.Header().Set("Access-Control-Allow-Credentials", "true")
}
@@ -341,8 +343,18 @@ func (t *userAgentTransport) RoundTrip(req *http.Request) (*http.Response, error
// httpClient returns an HTTP client that automatically adds the User-Agent header
func (s *Server) httpClient() *http.Client {
return userAgentHTTPClient(10 * time.Second)
}
// inferenceClient uses almost the same HTTP client, but without a timeout so
// long requests aren't truncated
func (s *Server) inferenceClient() *api.Client {
return api.NewClient(envconfig.Host(), userAgentHTTPClient(0))
}
func userAgentHTTPClient(timeout time.Duration) *http.Client {
return &http.Client{
Timeout: 10 * time.Second,
Timeout: timeout,
Transport: &userAgentTransport{
base: http.DefaultTransport,
},
@@ -720,11 +732,7 @@ func (s *Server) chat(w http.ResponseWriter, r *http.Request) error {
_, cancelLoading := context.WithCancel(ctx)
loading := false
c, err := api.ClientFromEnvironment()
if err != nil {
cancelLoading()
return err
}
c := s.inferenceClient()
// Check if the model exists locally by trying to show it
// TODO (jmorganca): skip this round trip and instead just act
@@ -1572,9 +1580,18 @@ func (s *Server) modelUpstream(w http.ResponseWriter, r *http.Request) error {
return json.NewEncoder(w).Encode(response)
}
n := model.ParseName(req.Model)
stale := true
if m, err := manifest.ParseNamedManifest(n); err == nil {
if m.Digest() == digest {
stale = false
} else if pushTime > 0 && m.FileInfo().ModTime().Unix() >= pushTime {
stale = false
}
}
response := responses.ModelUpstreamResponse{
Digest: digest,
PushTime: pushTime,
Stale: stale,
}
w.Header().Set("Content-Type", "application/json")
@@ -1672,7 +1689,6 @@ func supportsBrowserTools(model string) bool {
return strings.HasPrefix(strings.ToLower(model), "gpt-oss")
}
// buildChatRequest converts store.Chat to api.ChatRequest
func (s *Server) buildChatRequest(chat *store.Chat, model string, think any, availableTools []map[string]any) (*api.ChatRequest, error) {
var msgs []api.Message
+28
View File
@@ -15,6 +15,7 @@ import (
"sync/atomic"
"testing"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/app/store"
"github.com/ollama/ollama/app/updater"
)
@@ -526,6 +527,33 @@ func TestUserAgentTransport(t *testing.T) {
t.Logf("User-Agent transport successfully set: %s", receivedUA)
}
func TestInferenceClientUsesUserAgent(t *testing.T) {
var gotUserAgent atomic.Value
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotUserAgent.Store(r.Header.Get("User-Agent"))
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{}`))
}))
defer ts.Close()
t.Setenv("OLLAMA_HOST", ts.URL)
server := &Server{}
client := server.inferenceClient()
_, err := client.Show(context.Background(), &api.ShowRequest{Model: "test"})
if err != nil {
t.Fatalf("show request failed: %v", err)
}
receivedUA, _ := gotUserAgent.Load().(string)
expectedUA := userAgent()
if receivedUA != expectedUA {
t.Errorf("User-Agent mismatch\nExpected: %s\nReceived: %s", expectedUA, receivedUA)
}
}
func TestSupportsBrowserTools(t *testing.T) {
tests := []struct {
model string
+63 -35
View File
@@ -1,27 +1,31 @@
Ollama Benchmark Tool
---------------------
A Go-based command-line tool for benchmarking Ollama models with configurable parameters and multiple output formats.
A Go-based command-line tool for benchmarking Ollama models with configurable parameters, warmup phases, TTFT tracking, VRAM monitoring, and benchstat/CSV output.
## Features
* Benchmark multiple models in a single run
* Support for both text and image prompts
* Configurable generation parameters (temperature, max tokens, seed, etc.)
* Supports benchstat and CSV output formats
* Detailed performance metrics (prefill, generate, load, total durations)
* Warmup phase before timed epochs to stabilize measurements
* Time-to-first-token (TTFT) tracking per epoch
* Model metadata display (parameter size, quantization level, family)
* VRAM and CPU memory usage tracking via running process info
* Controlled prompt token length for reproducible benchmarks
* Benchstat and CSV output formats
## Building from Source
```
go build -o ollama-bench bench.go
./ollama-bench -model gpt-oss:20b -epochs 6 -format csv
go build -o ollama-bench ./cmd/bench
./ollama-bench -model gemma3 -epochs 6 -format csv
```
Using Go Run (without building)
```
go run bench.go -model gpt-oss:20b -epochs 3
go run ./cmd/bench -model gemma3 -epochs 3
```
## Usage
@@ -45,10 +49,16 @@ benchstat -col /name gemma.bench
./ollama-bench -model qwen3-vl -image photo.jpg -epochs 6 -max-tokens 100 -p "Describe this image"
```
### Controlled Prompt Length
```
./ollama-bench -model gemma3 -epochs 6 -prompt-tokens 512
```
### Advanced Example
```
./ollama-bench -model llama3 -epochs 10 -temperature 0.7 -max-tokens 500 -seed 42 -format csv -output results.csv
./ollama-bench -model llama3 -epochs 10 -temperature 0.7 -max-tokens 500 -seed 42 -warmup 2 -format csv -output results.csv
```
## Command Line Options
@@ -56,41 +66,48 @@ benchstat -col /name gemma.bench
| Option | Description | Default |
|----------|-------------|---------|
| -model | Comma-separated list of models to benchmark | (required) |
| -epochs | Number of iterations per model | 1 |
| -max-tokens | Maximum tokens for model response | 0 (unlimited) |
| -epochs | Number of iterations per model | 6 |
| -max-tokens | Maximum tokens for model response | 200 |
| -temperature | Temperature parameter | 0.0 |
| -seed | Random seed | 0 (random) |
| -timeout | Timeout in seconds | 300 |
| -p | Prompt text | "Write a long story." |
| -p | Prompt text | (default story prompt) |
| -image | Image file to include in prompt | |
| -k | Keep-alive duration in seconds | 0 |
| -format | Output format (benchstat, csv) | benchstat |
| -output | Output file for results | "" (stdout) |
| -warmup | Number of warmup requests before timing | 1 |
| -prompt-tokens | Generate prompt targeting ~N tokens (0 = use -p) | 0 |
| -v | Verbose mode | false |
| -debug | Show debug information | false |
## Output Formats
### Markdown Format
### Benchstat Format (default)
The default markdown format is suitable for copying and pasting into a GitHub issue and will look like:
```
Model | Step | Count | Duration | nsPerToken | tokensPerSec |
|-------|------|-------|----------|------------|--------------|
| gpt-oss:20b | prefill | 124 | 30.006458ms | 241987.56 | 4132.44 |
| gpt-oss:20b | generate | 200 | 2.646843954s | 13234219.77 | 75.56 |
| gpt-oss:20b | load | 1 | 121.674208ms | - | - |
| gpt-oss:20b | total | 1 | 2.861047625s | - | - |
```
### Benchstat Format
Compatible with Go's benchstat tool for statistical analysis:
Compatible with Go's benchstat tool for statistical analysis. Uses one value/unit pair per line, standard `ns/op` for timing metrics, and `ns/token` for throughput. Each epoch produces one set of lines -- benchstat aggregates across repeated runs to compute statistics.
```
BenchmarkModel/name=gpt-oss:20b/step=prefill 128 78125.00 ns/token 12800.00 token/sec
BenchmarkModel/name=gpt-oss:20b/step=generate 512 19531.25 ns/token 51200.00 token/sec
BenchmarkModel/name=gpt-oss:20b/step=load 1 1500000000 ns/request
# Model: gemma3 | Params: 4.3B | Quant: Q4_K_M | Family: gemma3 | Size: 4080218931 | VRAM: 4080218931
BenchmarkModel/name=gemma3/step=prefill 1 78125.00 ns/token 12800.00 token/sec
BenchmarkModel/name=gemma3/step=generate 1 19531.25 ns/token 51200.00 token/sec
BenchmarkModel/name=gemma3/step=ttft 1 45123000 ns/op
BenchmarkModel/name=gemma3/step=load 1 1500000000 ns/op
BenchmarkModel/name=gemma3/step=total 1 2861047625 ns/op
```
Use with benchstat:
```
./ollama-bench -model gemma3 -epochs 6 > gemma3.bench
benchstat -col /step gemma3.bench
```
Compare two runs:
```
./ollama-bench -model gemma3 -epochs 6 > before.bench
# ... make changes ...
./ollama-bench -model gemma3 -epochs 6 > after.bench
benchstat before.bench after.bench
```
### CSV Format
@@ -99,17 +116,28 @@ Machine-readable comma-separated values:
```
NAME,STEP,COUNT,NS_PER_COUNT,TOKEN_PER_SEC
gpt-oss:20b,prefill,128,78125.00,12800.00
gpt-oss:20b,generate,512,19531.25,51200.00
gpt-oss:20b,load,1,1500000000,0
# Model: gemma3 | Params: 4.3B | Quant: Q4_K_M | Family: gemma3 | Size: 4080218931 | VRAM: 4080218931
gemma3,prefill,128,78125.00,12800.00
gemma3,generate,512,19531.25,51200.00
gemma3,ttft,1,45123000,0
gemma3,load,1,1500000000,0
gemma3,total,1,2861047625,0
```
## Metrics Explained
The tool reports four types of metrics for each model:
The tool reports the following metrics for each epoch:
* prefill: Time spent processing the prompt
* generate: Time spent generating the response
* load: Model loading time (one-time cost)
* total: Total request duration
* **prefill**: Time spent processing the prompt (ns/token)
* **generate**: Time spent generating the response (ns/token)
* **ttft**: Time to first token -- latency from request start to first response content
* **load**: Model loading time (one-time cost)
* **total**: Total request duration
Additionally, the model info comment line (displayed once per model before epochs) includes:
* **Params**: Model parameter count (e.g., 4.3B)
* **Quant**: Quantization level (e.g., Q4_K_M)
* **Family**: Model family (e.g., gemma3)
* **Size**: Total model memory in bytes
* **VRAM**: GPU memory used by the loaded model (when Size > VRAM, the difference is CPU spill)
+371 -131
View File
@@ -17,19 +17,22 @@ import (
)
type flagOptions struct {
models *string
epochs *int
maxTokens *int
temperature *float64
seed *int
timeout *int
prompt *string
imageFile *string
keepAlive *float64
format *string
outputFile *string
debug *bool
verbose *bool
models *string
epochs *int
maxTokens *int
temperature *float64
seed *int
timeout *int
prompt *string
imageFile *string
keepAlive *float64
format *string
outputFile *string
debug *bool
verbose *bool
warmup *int
promptTokens *int
numCtx *int
}
type Metrics struct {
@@ -39,48 +42,203 @@ type Metrics struct {
Duration time.Duration
}
var once sync.Once
type ModelInfo struct {
Name string
ParameterSize string
QuantizationLevel string
Family string
SizeBytes int64
VRAMBytes int64
NumCtx int64
}
const DefaultPrompt = `Please write a descriptive story about a llama named Alonso who grows up to be President of the Land of Llamas. Include details about Alonso's childhood, adolescent years, and how he grew up to be a political mover and shaker. Write the story with a sense of whimsy.`
// Word list for generating prompts targeting a specific token count.
var promptWordList = []string{
"the", "quick", "brown", "fox", "jumps", "over", "lazy", "dog",
"a", "bright", "sunny", "day", "in", "the", "meadow", "where",
"flowers", "bloom", "and", "birds", "sing", "their", "morning",
"songs", "while", "gentle", "breeze", "carries", "sweet", "scent",
"of", "pine", "trees", "across", "rolling", "hills", "toward",
"distant", "mountains", "covered", "with", "fresh", "snow",
"beneath", "clear", "blue", "sky", "children", "play", "near",
"old", "stone", "bridge", "that", "crosses", "winding", "river",
}
// tokensPerWord is the calibrated ratio of tokens to words for the current model.
// Initialized with a heuristic, then updated during warmup based on actual tokenization.
var tokensPerWord = 1.3
func generatePromptForTokenCount(targetTokens int, epoch int) string {
targetWords := int(float64(targetTokens) / tokensPerWord)
if targetWords < 1 {
targetWords = 1
}
// Vary the starting offset by epoch to defeat KV cache prefix matching
offset := epoch * 7 // stride by a prime to get good distribution
n := len(promptWordList)
words := make([]string, targetWords)
for i := range words {
words[i] = promptWordList[((i+offset)%n+n)%n]
}
return strings.Join(words, " ")
}
// calibratePromptTokens adjusts tokensPerWord based on actual tokenization from a warmup run.
func calibratePromptTokens(targetTokens, actualTokens, wordCount int) {
if actualTokens <= 0 || wordCount <= 0 {
return
}
tokensPerWord = float64(actualTokens) / float64(wordCount)
newWords := int(float64(targetTokens) / tokensPerWord)
fmt.Fprintf(os.Stderr, "bench: calibrated %.2f tokens/word (target=%d, got=%d, words=%d → %d)\n",
tokensPerWord, targetTokens, actualTokens, wordCount, newWords)
}
func buildGenerateRequest(model string, fOpt flagOptions, imgData api.ImageData, epoch int) *api.GenerateRequest {
options := make(map[string]interface{})
if *fOpt.maxTokens > 0 {
options["num_predict"] = *fOpt.maxTokens
}
options["temperature"] = *fOpt.temperature
if fOpt.seed != nil && *fOpt.seed > 0 {
options["seed"] = *fOpt.seed
}
if fOpt.numCtx != nil && *fOpt.numCtx > 0 {
options["num_ctx"] = *fOpt.numCtx
}
var keepAliveDuration *api.Duration
if *fOpt.keepAlive > 0 {
duration := api.Duration{Duration: time.Duration(*fOpt.keepAlive * float64(time.Second))}
keepAliveDuration = &duration
}
prompt := *fOpt.prompt
if *fOpt.promptTokens > 0 {
prompt = generatePromptForTokenCount(*fOpt.promptTokens, epoch)
} else {
// Vary the prompt per epoch to defeat KV cache prefix matching
prompt = fmt.Sprintf("[%d] %s", epoch, prompt)
}
req := &api.GenerateRequest{
Model: model,
Prompt: prompt,
Raw: true,
Options: options,
KeepAlive: keepAliveDuration,
}
if imgData != nil {
req.Images = []api.ImageData{imgData}
}
return req
}
func fetchModelInfo(ctx context.Context, client *api.Client, model string) ModelInfo {
info := ModelInfo{Name: model}
resp, err := client.Show(ctx, &api.ShowRequest{Model: model})
if err != nil {
fmt.Fprintf(os.Stderr, "WARNING: Could not fetch model info for '%s': %v\n", model, err)
return info
}
info.ParameterSize = resp.Details.ParameterSize
info.QuantizationLevel = resp.Details.QuantizationLevel
info.Family = resp.Details.Family
return info
}
func fetchMemoryUsage(ctx context.Context, client *api.Client, model string) (size, vram int64) {
resp, err := client.ListRunning(ctx)
if err != nil {
if debug := os.Getenv("OLLAMA_DEBUG"); debug != "" {
fmt.Fprintf(os.Stderr, "WARNING: Could not fetch memory usage: %v\n", err)
}
return 0, 0
}
for _, m := range resp.Models {
if m.Name == model || m.Model == model {
return m.Size, m.SizeVRAM
}
}
for _, m := range resp.Models {
if strings.HasPrefix(m.Name, model) || strings.HasPrefix(m.Model, model) {
return m.Size, m.SizeVRAM
}
}
return 0, 0
}
func fetchContextLength(ctx context.Context, client *api.Client, model string) int64 {
resp, err := client.ListRunning(ctx)
if err != nil {
return 0
}
for _, m := range resp.Models {
if m.Name == model || m.Model == model || strings.HasPrefix(m.Name, model) || strings.HasPrefix(m.Model, model) {
return int64(m.ContextLength)
}
}
return 0
}
func outputFormatHeader(w io.Writer, format string, verbose bool) {
switch format {
case "benchstat":
if verbose {
fmt.Fprintf(w, "goos: %s\n", runtime.GOOS)
fmt.Fprintf(w, "goarch: %s\n", runtime.GOARCH)
}
case "csv":
headings := []string{"NAME", "STEP", "COUNT", "NS_PER_COUNT", "TOKEN_PER_SEC"}
fmt.Fprintln(w, strings.Join(headings, ","))
}
}
func outputModelInfo(w io.Writer, format string, info ModelInfo) {
params := cmp.Or(info.ParameterSize, "unknown")
quant := cmp.Or(info.QuantizationLevel, "unknown")
family := cmp.Or(info.Family, "unknown")
memStr := ""
if info.SizeBytes > 0 {
memStr = fmt.Sprintf(" | Size: %d | VRAM: %d", info.SizeBytes, info.VRAMBytes)
}
ctxStr := ""
if info.NumCtx > 0 {
ctxStr = fmt.Sprintf(" | NumCtx: %d", info.NumCtx)
}
fmt.Fprintf(w, "# Model: %s | Params: %s | Quant: %s | Family: %s%s%s\n",
info.Name, params, quant, family, memStr, ctxStr)
}
func OutputMetrics(w io.Writer, format string, metrics []Metrics, verbose bool) {
switch format {
case "benchstat":
if verbose {
printHeader := func() {
fmt.Fprintf(w, "sysname: %s\n", runtime.GOOS)
fmt.Fprintf(w, "machine: %s\n", runtime.GOARCH)
}
once.Do(printHeader)
}
for _, m := range metrics {
if m.Step == "generate" || m.Step == "prefill" {
if m.Count > 0 {
nsPerToken := float64(m.Duration.Nanoseconds()) / float64(m.Count)
tokensPerSec := float64(m.Count) / (float64(m.Duration.Nanoseconds()) + 1e-12) * 1e9
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=%s %d %.2f ns/token %.2f token/sec\n",
m.Model, m.Step, m.Count, nsPerToken, tokensPerSec)
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=%s 1 %.2f ns/token %.2f token/sec\n",
m.Model, m.Step, nsPerToken, tokensPerSec)
} else {
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=%s %d 0 ns/token 0 token/sec\n",
m.Model, m.Step, m.Count)
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=%s 1 0 ns/token 0 token/sec\n",
m.Model, m.Step)
}
} else if m.Step == "ttft" {
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=ttft 1 %d ns/op\n",
m.Model, m.Duration.Nanoseconds())
} else {
var suffix string
if m.Step == "load" {
suffix = "/step=load"
}
fmt.Fprintf(w, "BenchmarkModel/name=%s%s 1 %d ns/request\n",
m.Model, suffix, m.Duration.Nanoseconds())
fmt.Fprintf(w, "BenchmarkModel/name=%s/step=%s 1 %d ns/op\n",
m.Model, m.Step, m.Duration.Nanoseconds())
}
}
case "csv":
printHeader := func() {
headings := []string{"NAME", "STEP", "COUNT", "NS_PER_COUNT", "TOKEN_PER_SEC"}
fmt.Fprintln(w, strings.Join(headings, ","))
}
once.Do(printHeader)
for _, m := range metrics {
if m.Step == "generate" || m.Step == "prefill" {
var nsPerToken float64
@@ -94,39 +252,14 @@ func OutputMetrics(w io.Writer, format string, metrics []Metrics, verbose bool)
fmt.Fprintf(w, "%s,%s,1,%d,0\n", m.Model, m.Step, m.Duration.Nanoseconds())
}
}
case "markdown":
printHeader := func() {
fmt.Fprintln(w, "| Model | Step | Count | Duration | nsPerToken | tokensPerSec |")
fmt.Fprintln(w, "|-------|------|-------|----------|------------|--------------|")
}
once.Do(printHeader)
for _, m := range metrics {
var nsPerToken, tokensPerSec float64
var nsPerTokenStr, tokensPerSecStr string
if m.Step == "generate" || m.Step == "prefill" {
nsPerToken = float64(m.Duration.Nanoseconds()) / float64(m.Count)
tokensPerSec = float64(m.Count) / (float64(m.Duration.Nanoseconds()) + 1e-12) * 1e9
nsPerTokenStr = fmt.Sprintf("%.2f", nsPerToken)
tokensPerSecStr = fmt.Sprintf("%.2f", tokensPerSec)
} else {
nsPerTokenStr = "-"
tokensPerSecStr = "-"
}
fmt.Fprintf(w, "| %s | %s | %d | %v | %s | %s |\n",
m.Model, m.Step, m.Count, m.Duration, nsPerTokenStr, tokensPerSecStr)
}
default:
fmt.Fprintf(os.Stderr, "Unknown output format '%s'\n", format)
}
}
func BenchmarkChat(fOpt flagOptions) error {
func BenchmarkModel(fOpt flagOptions) error {
models := strings.Split(*fOpt.models, ",")
// todo - add multi-image support
var imgData api.ImageData
var err error
if *fOpt.imageFile != "" {
@@ -158,71 +291,141 @@ func BenchmarkChat(fOpt flagOptions) error {
out = f
}
outputFormatHeader(out, *fOpt.format, *fOpt.verbose)
// Log prompt-tokens info in debug mode
if *fOpt.debug && *fOpt.promptTokens > 0 {
prompt := generatePromptForTokenCount(*fOpt.promptTokens, 0)
wordCount := len(strings.Fields(prompt))
fmt.Fprintf(os.Stderr, "Generated prompt targeting ~%d tokens (%d words, varied per epoch)\n", *fOpt.promptTokens, wordCount)
}
for _, model := range models {
for range *fOpt.epochs {
options := make(map[string]interface{})
if *fOpt.maxTokens > 0 {
options["num_predict"] = *fOpt.maxTokens
}
options["temperature"] = *fOpt.temperature
if fOpt.seed != nil && *fOpt.seed > 0 {
options["seed"] = *fOpt.seed
}
var keepAliveDuration *api.Duration
if *fOpt.keepAlive > 0 {
duration := api.Duration{Duration: time.Duration(*fOpt.keepAlive * float64(time.Second))}
keepAliveDuration = &duration
}
req := &api.ChatRequest{
Model: model,
Messages: []api.Message{
{
Role: "user",
Content: *fOpt.prompt,
},
},
Options: options,
KeepAlive: keepAliveDuration,
}
if imgData != nil {
req.Messages[0].Images = []api.ImageData{imgData}
}
var responseMetrics *api.Metrics
// Fetch model info
infoCtx, infoCancel := context.WithTimeout(context.Background(), 10*time.Second)
info := fetchModelInfo(infoCtx, client, model)
infoCancel()
// Warmup phase (uses negative epoch numbers to avoid colliding with timed epochs)
for i := range *fOpt.warmup {
req := buildGenerateRequest(model, fOpt, imgData, -(i + 1))
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(*fOpt.timeout)*time.Second)
defer cancel()
err = client.Chat(ctx, req, func(resp api.ChatResponse) error {
if *fOpt.debug {
fmt.Fprintf(os.Stderr, "%s", cmp.Or(resp.Message.Thinking, resp.Message.Content))
}
var warmupMetrics *api.Metrics
err = client.Generate(ctx, req, func(resp api.GenerateResponse) error {
if resp.Done {
responseMetrics = &resp.Metrics
warmupMetrics = &resp.Metrics
}
return nil
})
if *fOpt.debug {
fmt.Fprintln(os.Stderr)
}
cancel()
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
fmt.Fprintf(os.Stderr, "ERROR: Chat request timed out with model '%s' after %vs\n", model, 1)
continue
fmt.Fprintf(os.Stderr, "WARNING: Warmup %d/%d for %s failed: %v\n", i+1, *fOpt.warmup, model, err)
} else {
if *fOpt.debug {
fmt.Fprintf(os.Stderr, "Warmup %d/%d for %s complete\n", i+1, *fOpt.warmup, model)
}
fmt.Fprintf(os.Stderr, "ERROR: Couldn't chat with model '%s': %v\n", model, err)
// Calibrate prompt token count on last warmup run
if i == *fOpt.warmup-1 && *fOpt.promptTokens > 0 && warmupMetrics != nil {
prompt := generatePromptForTokenCount(*fOpt.promptTokens, -(i + 1))
wordCount := len(strings.Fields(prompt))
calibratePromptTokens(*fOpt.promptTokens, warmupMetrics.PromptEvalCount, wordCount)
}
}
}
// Fetch memory/context info once after warmup (model is loaded and stable)
memCtx, memCancel := context.WithTimeout(context.Background(), 5*time.Second)
info.SizeBytes, info.VRAMBytes = fetchMemoryUsage(memCtx, client, model)
if fOpt.numCtx != nil && *fOpt.numCtx > 0 {
info.NumCtx = int64(*fOpt.numCtx)
} else {
info.NumCtx = fetchContextLength(memCtx, client, model)
}
memCancel()
outputModelInfo(out, *fOpt.format, info)
// Timed epoch loop
shortCount := 0
for epoch := range *fOpt.epochs {
var responseMetrics *api.Metrics
var ttft time.Duration
short := false
// Retry loop: if the model hits a stop token before max-tokens,
// retry with a different prompt (up to maxRetries times).
const maxRetries = 3
for attempt := range maxRetries + 1 {
responseMetrics = nil
ttft = 0
var ttftOnce sync.Once
req := buildGenerateRequest(model, fOpt, imgData, epoch+attempt*1000)
requestStart := time.Now()
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(*fOpt.timeout)*time.Second)
err = client.Generate(ctx, req, func(resp api.GenerateResponse) error {
if *fOpt.debug {
fmt.Fprintf(os.Stderr, "%s", cmp.Or(resp.Thinking, resp.Response))
}
// Capture TTFT on first content
ttftOnce.Do(func() {
if resp.Response != "" || resp.Thinking != "" {
ttft = time.Since(requestStart)
}
})
if resp.Done {
responseMetrics = &resp.Metrics
}
return nil
})
cancel()
if *fOpt.debug {
fmt.Fprintln(os.Stderr)
}
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
fmt.Fprintf(os.Stderr, "ERROR: Request timed out with model '%s' after %vs\n", model, *fOpt.timeout)
} else {
fmt.Fprintf(os.Stderr, "ERROR: Couldn't generate with model '%s': %v\n", model, err)
}
break
}
if responseMetrics == nil {
fmt.Fprintf(os.Stderr, "ERROR: No metrics received for model '%s'\n", model)
break
}
// Check if the response was shorter than requested
short = *fOpt.maxTokens > 0 && responseMetrics.EvalCount < *fOpt.maxTokens
if !short || attempt == maxRetries {
break
}
if *fOpt.debug {
fmt.Fprintf(os.Stderr, "Short response (%d/%d tokens), retrying with different prompt (attempt %d/%d)\n",
responseMetrics.EvalCount, *fOpt.maxTokens, attempt+1, maxRetries)
}
}
if err != nil || responseMetrics == nil {
continue
}
if responseMetrics == nil {
fmt.Fprintf(os.Stderr, "ERROR: No metrics received for model '%s'\n", model)
continue
if short {
shortCount++
if *fOpt.debug {
fmt.Fprintf(os.Stderr, "WARNING: Short response (%d/%d tokens) after %d retries for epoch %d\n",
responseMetrics.EvalCount, *fOpt.maxTokens, maxRetries, epoch+1)
}
}
metrics := []Metrics{
@@ -238,6 +441,12 @@ func BenchmarkChat(fOpt flagOptions) error {
Count: responseMetrics.EvalCount,
Duration: responseMetrics.EvalDuration,
},
{
Model: model,
Step: "ttft",
Count: 1,
Duration: ttft,
},
{
Model: model,
Step: "load",
@@ -254,15 +463,42 @@ func BenchmarkChat(fOpt flagOptions) error {
OutputMetrics(out, *fOpt.format, metrics, *fOpt.verbose)
if *fOpt.debug && *fOpt.promptTokens > 0 {
fmt.Fprintf(os.Stderr, "Generated prompt targeting ~%d tokens (actual: %d)\n",
*fOpt.promptTokens, responseMetrics.PromptEvalCount)
}
if *fOpt.keepAlive > 0 {
time.Sleep(time.Duration(*fOpt.keepAlive*float64(time.Second)) + 200*time.Millisecond)
}
}
if shortCount > 0 {
fmt.Fprintf(os.Stderr, "WARNING: %d/%d epochs for '%s' had short responses (<%d tokens). Generation metrics may be unreliable.\n",
shortCount, *fOpt.epochs, model, *fOpt.maxTokens)
}
// Unload model before moving to the next one
unloadModel(client, model, *fOpt.timeout)
}
return nil
}
func unloadModel(client *api.Client, model string, timeout int) {
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeout)*time.Second)
defer cancel()
zero := api.Duration{Duration: 0}
req := &api.GenerateRequest{
Model: model,
KeepAlive: &zero,
}
_ = client.Generate(ctx, req, func(resp api.GenerateResponse) error {
return nil
})
}
func readImage(filePath string) (api.ImageData, error) {
file, err := os.Open(filePath)
if err != nil {
@@ -280,19 +516,22 @@ func readImage(filePath string) (api.ImageData, error) {
func main() {
fOpt := flagOptions{
models: flag.String("model", "", "Model to benchmark"),
epochs: flag.Int("epochs", 6, "Number of epochs (iterations) per model"),
maxTokens: flag.Int("max-tokens", 200, "Maximum tokens for model response"),
temperature: flag.Float64("temperature", 0, "Temperature parameter"),
seed: flag.Int("seed", 0, "Random seed"),
timeout: flag.Int("timeout", 60*5, "Timeout in seconds (default 300s)"),
prompt: flag.String("p", DefaultPrompt, "Prompt to use"),
imageFile: flag.String("image", "", "Filename for an image to include"),
keepAlive: flag.Float64("k", 0, "Keep alive duration in seconds"),
format: flag.String("format", "markdown", "Output format [benchstat|csv] (default benchstat)"),
outputFile: flag.String("output", "", "Output file for results (stdout if empty)"),
verbose: flag.Bool("v", false, "Show system information"),
debug: flag.Bool("debug", false, "Show debug information"),
models: flag.String("model", "", "Model to benchmark"),
epochs: flag.Int("epochs", 6, "Number of epochs (iterations) per model"),
maxTokens: flag.Int("max-tokens", 200, "Maximum tokens for model response"),
temperature: flag.Float64("temperature", 0, "Temperature parameter"),
seed: flag.Int("seed", 0, "Random seed"),
timeout: flag.Int("timeout", 60*5, "Timeout in seconds (default 300s)"),
prompt: flag.String("p", DefaultPrompt, "Prompt to use"),
imageFile: flag.String("image", "", "Filename for an image to include"),
keepAlive: flag.Float64("k", 0, "Keep alive duration in seconds"),
format: flag.String("format", "benchstat", "Output format [benchstat|csv]"),
outputFile: flag.String("output", "", "Output file for results (stdout if empty)"),
verbose: flag.Bool("v", false, "Show system information"),
debug: flag.Bool("debug", false, "Show debug information"),
warmup: flag.Int("warmup", 1, "Number of warmup requests before timing"),
promptTokens: flag.Int("prompt-tokens", 0, "Generate prompt targeting ~N tokens (0 = use -p prompt)"),
numCtx: flag.Int("num-ctx", 0, "Context size (0 = server default)"),
}
flag.Usage = func() {
@@ -302,11 +541,12 @@ func main() {
fmt.Fprintf(os.Stderr, "Options:\n")
flag.PrintDefaults()
fmt.Fprintf(os.Stderr, "\nExamples:\n")
fmt.Fprintf(os.Stderr, " bench -model gpt-oss:20b -epochs 3 -temperature 0.7\n")
fmt.Fprintf(os.Stderr, " bench -model gemma3,llama3 -epochs 6\n")
fmt.Fprintf(os.Stderr, " bench -model gemma3 -epochs 6 -prompt-tokens 512 -format csv\n")
}
flag.Parse()
if !slices.Contains([]string{"markdown", "benchstat", "csv"}, *fOpt.format) {
if !slices.Contains([]string{"benchstat", "csv"}, *fOpt.format) {
fmt.Fprintf(os.Stderr, "ERROR: Unknown format '%s'\n", *fOpt.format)
os.Exit(1)
}
@@ -317,5 +557,5 @@ func main() {
return
}
BenchmarkChat(fOpt)
BenchmarkModel(fOpt)
}
+1088 -141
View File
File diff suppressed because it is too large. Load diff
+437 -267
View File
@@ -11,6 +11,7 @@ import (
"fmt"
"io"
"log"
"log/slog"
"math"
"net"
"net/http"
@@ -38,9 +39,12 @@ import (
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/launch"
"github.com/ollama/ollama/cmd/tui"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/format"
"github.com/ollama/ollama/internal/modelref"
"github.com/ollama/ollama/logutil"
"github.com/ollama/ollama/parser"
"github.com/ollama/ollama/progress"
"github.com/ollama/ollama/readline"
@@ -50,46 +54,84 @@ import (
"github.com/ollama/ollama/types/syncmap"
"github.com/ollama/ollama/version"
xcmd "github.com/ollama/ollama/x/cmd"
"github.com/ollama/ollama/x/create"
xcreate "github.com/ollama/ollama/x/create"
xcreateclient "github.com/ollama/ollama/x/create/client"
"github.com/ollama/ollama/x/imagegen"
)
func init() {
// Override default selectors to use Bubbletea TUI instead of raw terminal I/O.
config.DefaultSingleSelector = func(title string, items []config.ModelItem, current string) (string, error) {
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectSingle(title, tuiItems, current)
if errors.Is(err, tui.ErrCancelled) {
return "", config.ErrCancelled
}
return result, err
launch.DefaultSingleSelector = func(title string, items []launch.SelectionItem, current string) (string, error) {
return runTUISingleSelector(title, items, current, nil)
}
config.DefaultMultiSelector = func(title string, items []config.ModelItem, preChecked []string) ([]string, error) {
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectMultiple(title, tuiItems, preChecked)
if errors.Is(err, tui.ErrCancelled) {
return nil, config.ErrCancelled
}
return result, err
launch.DefaultSingleSelectorWithUpdates = func(title string, items []launch.SelectionItem, current string, updates <-chan []launch.SelectionItem) (string, error) {
return runTUISingleSelector(title, items, current, updates)
}
config.DefaultSignIn = func(modelName, signInURL string) (string, error) {
launch.DefaultMultiSelector = func(title string, items []launch.SelectionItem, preChecked []string) ([]string, error) {
return runTUIMultiSelector(title, items, preChecked, nil)
}
launch.DefaultMultiSelectorWithUpdates = func(title string, items []launch.SelectionItem, preChecked []string, updates <-chan []launch.SelectionItem) ([]string, error) {
return runTUIMultiSelector(title, items, preChecked, updates)
}
launch.DefaultSignIn = func(modelName, signInURL string) (string, error) {
userName, err := tui.RunSignIn(modelName, signInURL)
if errors.Is(err, tui.ErrCancelled) {
return "", config.ErrCancelled
return "", launch.ErrCancelled
}
return userName, err
}
config.DefaultConfirmPrompt = func(prompt string) (bool, error) {
ok, err := tui.RunConfirm(prompt)
launch.DefaultUpgrade = func(modelName, requiredPlan string) (string, error) {
plan, err := tui.RunUpgrade(modelName, requiredPlan)
if errors.Is(err, tui.ErrCancelled) {
return false, config.ErrCancelled
return "", launch.ErrCancelled
}
return ok, err
return plan, err
}
launch.DefaultConfirmPrompt = tui.RunConfirmWithOptions
}
func runTUISingleSelector(title string, items []launch.SelectionItem, current string, updates <-chan []launch.SelectionItem) (string, error) {
if !term.IsTerminal(int(os.Stdin.Fd())) || !term.IsTerminal(int(os.Stdout.Fd())) {
return "", fmt.Errorf("model selection requires an interactive terminal; use --model to run in headless mode")
}
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectSingleWithUpdates(title, tuiItems, current, convertSelectionItemUpdates(updates))
if errors.Is(err, tui.ErrCancelled) {
return "", launch.ErrCancelled
}
return result, err
}
func runTUIMultiSelector(title string, items []launch.SelectionItem, preChecked []string, updates <-chan []launch.SelectionItem) ([]string, error) {
if !term.IsTerminal(int(os.Stdin.Fd())) || !term.IsTerminal(int(os.Stdout.Fd())) {
return nil, fmt.Errorf("model selection requires an interactive terminal; use --model to run in headless mode")
}
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectMultipleWithUpdates(title, tuiItems, preChecked, convertSelectionItemUpdates(updates))
if errors.Is(err, tui.ErrCancelled) {
return nil, launch.ErrCancelled
}
return result, err
}
func convertSelectionItemUpdates(updates <-chan []launch.SelectionItem) <-chan []tui.SelectItem {
if updates == nil {
return nil
}
out := make(chan []tui.SelectItem, 1)
go func() {
defer close(out)
for items := range updates {
out <- tui.ReorderItems(tui.ConvertItems(items))
}
}()
return out
}
const ConnectInstructions = "If your browser did not open, navigate to:\n %s\n\n"
@@ -131,6 +173,50 @@ func getModelfileName(cmd *cobra.Command) (string, error) {
return absName, nil
}
// isLocalhost returns true if the configured Ollama host is a loopback or unspecified address.
func isLocalhost() bool {
host := envconfig.Host()
h, _, _ := net.SplitHostPort(host.Host)
if h == "localhost" {
return true
}
ip := net.ParseIP(h)
return ip != nil && (ip.IsLoopback() || ip.IsUnspecified())
}
func resolveExperimentalLocalModelDir(ref, filename string) string {
if ref == "" || filepath.IsAbs(ref) || filename == "" {
return ref
}
candidate := filepath.Join(filepath.Dir(filename), ref)
if xcreate.IsSafetensorsModelDir(candidate) || xcreate.IsTensorModelDir(candidate) {
return candidate
}
return ref
}
func resolveExperimentalDraftDir(ref, filename string) (string, error) {
if ref == "" {
return "", nil
}
if filepath.IsAbs(ref) {
if xcreate.IsSafetensorsModelDir(ref) {
return ref, nil
}
return "", fmt.Errorf("draft %s is not a supported safetensors model directory", ref)
}
if filename != "" {
candidate := filepath.Join(filepath.Dir(filename), ref)
if xcreate.IsSafetensorsModelDir(candidate) {
return candidate, nil
}
}
return "", fmt.Errorf("DRAFT model references are not supported with --experimental yet: %s", ref)
}
func CreateHandler(cmd *cobra.Command, args []string) error {
p := progress.NewProgress(os.Stderr)
defer p.Stop()
@@ -143,8 +229,17 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
}
// Check for --experimental flag for safetensors model creation
// This gates both safetensors LLM and imagegen model creation
experimental, _ := cmd.Flags().GetBool("experimental")
draftQuantize, _ := cmd.Flags().GetString("draft-quantize")
if draftQuantize != "" && !experimental {
return errors.New("--draft-quantize requires --experimental")
}
if experimental {
if !isLocalhost() {
return errors.New("remote safetensor model creation not yet supported")
}
// Get Modelfile content - either from -f flag or default to "FROM ."
var reader io.Reader
filename, err := getModelfileName(cmd)
@@ -168,59 +263,36 @@ func CreateHandler(cmd *cobra.Command, args []string) error {
return fmt.Errorf("failed to parse Modelfile: %w", err)
}
// Extract FROM path and configuration
var modelDir string
mfConfig := &xcreateclient.ModelfileConfig{}
modelDir, mfConfig, err := xcreateclient.ConfigFromModelfile(modelfile)
if err != nil {
return err
}
for _, cmd := range modelfile.Commands {
switch cmd.Name {
case "model":
modelDir = cmd.Args
case "template":
mfConfig.Template = cmd.Args
case "system":
mfConfig.System = cmd.Args
case "license":
mfConfig.License = cmd.Args
case "parser":
mfConfig.Parser = cmd.Args
case "renderer":
mfConfig.Renderer = cmd.Args
modelDir = resolveExperimentalLocalModelDir(modelDir, filename)
if mfConfig.Draft != "" {
draftDir, err := resolveExperimentalDraftDir(mfConfig.Draft, filename)
if err != nil {
return err
}
}
if modelDir == "" {
modelDir = "."
}
// Resolve relative paths based on Modelfile location
if !filepath.IsAbs(modelDir) && filename != "" {
modelDir = filepath.Join(filepath.Dir(filename), modelDir)
mfConfig.Draft = draftDir
}
quantize, _ := cmd.Flags().GetString("quantize")
return xcreateclient.CreateModel(xcreateclient.CreateOptions{
ModelName: modelName,
ModelDir: modelDir,
Quantize: quantize,
Modelfile: mfConfig,
ModelName: modelName,
ModelDir: modelDir,
Quantize: quantize,
DraftQuantize: draftQuantize,
Modelfile: mfConfig,
}, p)
}
// Standard Modelfile + API path
var reader io.Reader
filename, err := getModelfileName(cmd)
if os.IsNotExist(err) {
if filename == "" {
// No Modelfile found - check if current directory is an image gen model
if create.IsTensorModelDir(".") {
quantize, _ := cmd.Flags().GetString("quantize")
return xcreateclient.CreateModel(xcreateclient.CreateOptions{
ModelName: modelName,
ModelDir: ".",
Quantize: quantize,
}, p)
}
reader = strings.NewReader("FROM .\n")
} else {
return errModelfileNotFound
@@ -406,12 +478,14 @@ func loadOrUnloadModel(cmd *cobra.Command, opts *runOptions) error {
return err
}
requestedCloud := modelref.HasExplicitCloudSource(opts.Model)
if info, err := client.Show(cmd.Context(), &api.ShowRequest{Model: opts.Model}); err != nil {
return err
} else if info.RemoteHost != "" {
} else if info.RemoteHost != "" || requestedCloud {
// Cloud model, no need to load/unload
isCloud := strings.HasPrefix(info.RemoteHost, "https://ollama.com")
isCloud := requestedCloud || strings.HasPrefix(info.RemoteHost, "https://ollama.com")
// Check if user is signed in for ollama.com cloud models
if isCloud {
@@ -422,10 +496,14 @@ func loadOrUnloadModel(cmd *cobra.Command, opts *runOptions) error {
if opts.ShowConnect {
p.StopAndClear()
remoteModel := info.RemoteModel
if remoteModel == "" {
remoteModel = opts.Model
}
if isCloud {
fmt.Fprintf(os.Stderr, "Connecting to '%s' on 'ollama.com' ⚡\n", info.RemoteModel)
fmt.Fprintf(os.Stderr, "Connecting to '%s' on 'ollama.com' ⚡\n", remoteModel)
} else {
fmt.Fprintf(os.Stderr, "Connecting to '%s' on '%s'\n", info.RemoteModel, info.RemoteHost)
fmt.Fprintf(os.Stderr, "Connecting to '%s' on '%s'\n", remoteModel, info.RemoteHost)
}
}
@@ -497,6 +575,64 @@ func generateEmbedding(cmd *cobra.Command, modelName, input string, keepAlive *a
return nil
}
// TODO(parthsareen): consolidate with TUI signin flow
func handleCloudAuthorizationError(err error) bool {
var authErr api.AuthorizationError
if errors.As(err, &authErr) && authErr.StatusCode == http.StatusUnauthorized {
fmt.Printf("You need to be signed in to Ollama to run Cloud models.\n\n")
if authErr.SigninURL != "" {
fmt.Printf(ConnectInstructions, authErr.SigninURL)
}
return true
}
return false
}
// TEMP(drifkin): To match legacy `ollama run some-model:cloud` behavior, we
// best-effort pull cloud stub files for any explicit cloud source models.
// Remove this once `/api/tags` is cloud-aware.
func ensureCloudStub(ctx context.Context, client *api.Client, modelName string) {
if !modelref.HasExplicitCloudSource(modelName) {
return
}
normalizedName, _, err := modelref.NormalizePullName(modelName)
if err != nil {
slog.Warn("failed to normalize pull name", "model", modelName, "error", err, "normalizedName", normalizedName)
return
}
listResp, err := client.List(ctx)
if err != nil {
slog.Warn("failed to list models", "error", err)
return
}
if hasListedModelName(listResp.Models, modelName) || hasListedModelName(listResp.Models, normalizedName) {
return
}
logutil.Trace("pulling cloud stub", "model", modelName, "normalizedName", normalizedName)
err = client.Pull(ctx, &api.PullRequest{
Model: normalizedName,
}, func(api.ProgressResponse) error {
return nil
})
if err != nil {
slog.Warn("failed to pull cloud stub", "model", modelName, "error", err)
}
}
func hasListedModelName(models []api.ListModelResponse, name string) bool {
for _, m := range models {
if strings.EqualFold(m.Name, name) || strings.EqualFold(m.Model, name) {
return true
}
}
return false
}
func RunHandler(cmd *cobra.Command, args []string) error {
interactive := true
@@ -527,10 +663,10 @@ func RunHandler(cmd *cobra.Command, args []string) error {
opts.Think = &api.ThinkValue{Value: true}
case "false":
opts.Think = &api.ThinkValue{Value: false}
case "high", "medium", "low":
case "high", "medium", "low", "max":
opts.Think = &api.ThinkValue{Value: thinkStr}
default:
return fmt.Errorf("invalid value for --think: %q (must be true, false, high, medium, or low)", thinkStr)
return fmt.Errorf("invalid value for --think: %q (must be true, false, high, medium, low, or max)", thinkStr)
}
} else {
opts.Think = nil
@@ -585,17 +721,6 @@ func RunHandler(cmd *cobra.Command, args []string) error {
}
opts.WordWrap = !nowrap
useImagegen := false
if cmd.Flags().Lookup("imagegen") != nil {
useImagegen, err = cmd.Flags().GetBool("imagegen")
if err != nil {
return err
}
}
if useImagegen {
opts.Options["use_imagegen_runner"] = true
}
// Fill out the rest of the options based on information about the
// model.
client, err := api.ClientFromEnvironment()
@@ -604,12 +729,16 @@ func RunHandler(cmd *cobra.Command, args []string) error {
}
name := args[0]
requestedCloud := modelref.HasExplicitCloudSource(name)
info, err := func() (*api.ShowResponse, error) {
showReq := &api.ShowRequest{Name: name}
info, err := client.Show(cmd.Context(), showReq)
var se api.StatusError
if errors.As(err, &se) && se.StatusCode == http.StatusNotFound {
if requestedCloud {
return nil, err
}
if err := PullHandler(cmd, []string{name}); err != nil {
return nil, err
}
@@ -618,15 +747,21 @@ func RunHandler(cmd *cobra.Command, args []string) error {
return info, err
}()
if err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return err
}
ensureCloudStub(cmd.Context(), client, name)
opts.Think, err = inferThinkingOption(&info.Capabilities, &opts, thinkFlag.Changed)
if err != nil {
return err
}
opts.MultiModal = slices.Contains(info.Capabilities, model.CapabilityVision)
audioCapable := slices.Contains(info.Capabilities, model.CapabilityAudio)
opts.MultiModal = slices.Contains(info.Capabilities, model.CapabilityVision) || audioCapable
// TODO: remove the projector info and vision info checks below,
// these are left in for backwards compatibility with older servers
@@ -641,7 +776,7 @@ func RunHandler(cmd *cobra.Command, args []string) error {
}
}
opts.ParentModel = info.Details.ParentModel
applyShowResponseToRunOptions(&opts, info)
// Check if this is an embedding model
isEmbeddingModel := slices.Contains(info.Capabilities, model.CapabilityEmbedding)
@@ -712,7 +847,13 @@ func RunHandler(cmd *cobra.Command, args []string) error {
return generateInteractive(cmd, opts)
}
return generate(cmd, opts)
if err := generate(cmd, opts); err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return err
}
return nil
}
func SigninHandler(cmd *cobra.Command, args []string) error {
@@ -1351,23 +1492,30 @@ func PullHandler(cmd *cobra.Command, args []string) error {
type generateContextKey string
type runOptions struct {
Model string
ParentModel string
Prompt string
Messages []api.Message
WordWrap bool
Format string
System string
Images []api.ImageData
Options map[string]any
MultiModal bool
KeepAlive *api.Duration
Think *api.ThinkValue
HideThinking bool
ShowConnect bool
Model string
ParentModel string
LoadedMessages []api.Message
Prompt string
Messages []api.Message
WordWrap bool
Format string
System string
Images []api.ImageData
Options map[string]any
MultiModal bool
KeepAlive *api.Duration
Think *api.ThinkValue
HideThinking bool
ShowConnect bool
}
func (r runOptions) Copy() runOptions {
var loadedMessages []api.Message
if r.LoadedMessages != nil {
loadedMessages = make([]api.Message, len(r.LoadedMessages))
copy(loadedMessages, r.LoadedMessages)
}
var messages []api.Message
if r.Messages != nil {
messages = make([]api.Message, len(r.Messages))
@@ -1395,23 +1543,29 @@ func (r runOptions) Copy() runOptions {
}
return runOptions{
Model: r.Model,
ParentModel: r.ParentModel,
Prompt: r.Prompt,
Messages: messages,
WordWrap: r.WordWrap,
Format: r.Format,
System: r.System,
Images: images,
Options: opts,
MultiModal: r.MultiModal,
KeepAlive: r.KeepAlive,
Think: think,
HideThinking: r.HideThinking,
ShowConnect: r.ShowConnect,
Model: r.Model,
ParentModel: r.ParentModel,
LoadedMessages: loadedMessages,
Prompt: r.Prompt,
Messages: messages,
WordWrap: r.WordWrap,
Format: r.Format,
System: r.System,
Images: images,
Options: opts,
MultiModal: r.MultiModal,
KeepAlive: r.KeepAlive,
Think: think,
HideThinking: r.HideThinking,
ShowConnect: r.ShowConnect,
}
}
func applyShowResponseToRunOptions(opts *runOptions, info *api.ShowResponse) {
opts.ParentModel = info.Details.ParentModel
opts.LoadedMessages = slices.Clone(info.Messages)
}
type displayResponseState struct {
lineLength int
wordBuffer string
@@ -1419,6 +1573,9 @@ type displayResponseState struct {
func displayResponse(content string, wordWrap bool, state *displayResponseState) {
termWidth, _, _ := term.GetSize(int(os.Stdout.Fd()))
if termWidth == 0 {
termWidth = 80
}
if wordWrap && termWidth >= 10 {
for _, ch := range content {
if state.lineLength+1 > termWidth-5 {
@@ -1892,6 +2049,77 @@ func ensureServerRunning(ctx context.Context) error {
}
}
func launchInteractiveModel(cmd *cobra.Command, modelName string) error {
opts := runOptions{
Model: modelName,
WordWrap: os.Getenv("TERM") == "xterm-256color",
Options: map[string]any{},
ShowConnect: true,
}
client, err := api.ClientFromEnvironment()
if err != nil {
return err
}
requestedCloud := modelref.HasExplicitCloudSource(modelName)
info, err := func() (*api.ShowResponse, error) {
showReq := &api.ShowRequest{Name: modelName}
info, err := client.Show(cmd.Context(), showReq)
var se api.StatusError
if errors.As(err, &se) && se.StatusCode == http.StatusNotFound {
if requestedCloud {
return nil, err
}
if err := PullHandler(cmd, []string{modelName}); err != nil {
return nil, err
}
return client.Show(cmd.Context(), &api.ShowRequest{Name: modelName})
}
return info, err
}()
if err != nil {
if handleCloudAuthorizationError(err) {
return nil
}
return err
}
ensureCloudStub(cmd.Context(), client, modelName)
opts.Think, err = inferThinkingOption(&info.Capabilities, &opts, false)
if err != nil {
return err
}
audioCapable := slices.Contains(info.Capabilities, model.CapabilityAudio)
opts.MultiModal = slices.Contains(info.Capabilities, model.CapabilityVision) || audioCapable
// TODO: remove the projector info and vision info checks below,
// these are left in for backwards compatibility with older servers
// that don't have the capabilities field in the model info
if len(info.ProjectorInfo) != 0 {
opts.MultiModal = true
}
for k := range info.ModelInfo {
if strings.Contains(k, ".vision.") {
opts.MultiModal = true
break
}
}
applyShowResponseToRunOptions(&opts, info)
if err := loadOrUnloadModel(cmd, &opts); err != nil {
return fmt.Errorf("error loading model: %w", err)
}
if err := generateInteractive(cmd, opts); err != nil {
return fmt.Errorf("error running model: %w", err)
}
return nil
}
// runInteractiveTUI runs the main interactive TUI menu.
func runInteractiveTUI(cmd *cobra.Command) {
// Ensure the server is running before showing the TUI
@@ -1900,175 +2128,113 @@ func runInteractiveTUI(cmd *cobra.Command) {
return
}
// Selector adapters for tui
singleSelector := func(title string, items []config.ModelItem, current string) (string, error) {
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectSingle(title, tuiItems, current)
if errors.Is(err, tui.ErrCancelled) {
return "", config.ErrCancelled
}
return result, err
}
multiSelector := func(title string, items []config.ModelItem, preChecked []string) ([]string, error) {
tuiItems := tui.ReorderItems(tui.ConvertItems(items))
result, err := tui.SelectMultiple(title, tuiItems, preChecked)
if errors.Is(err, tui.ErrCancelled) {
return nil, config.ErrCancelled
}
return result, err
accountPrefetch := launch.StartAccountStatePrefetch(cmd.Context())
deps := launcherDeps{
buildState: launch.BuildLauncherState,
runMenu: tui.RunMenu,
resolveRunModel: launch.ResolveRunModel,
launchIntegration: launch.LaunchIntegration,
runModel: launchInteractiveModel,
accountState: accountPrefetch.StateIfReady,
accountStateUpdates: accountPrefetch.StateUpdates,
}
for {
result, err := tui.Run()
continueLoop, err := runInteractiveTUIStep(cmd, deps)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
}
if !continueLoop {
return
}
}
}
runModel := func(modelName string) {
client, err := api.ClientFromEnvironment()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
return
}
if err := config.ShowOrPull(cmd.Context(), client, modelName); err != nil {
if errors.Is(err, config.ErrCancelled) {
return
}
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
return
}
_ = config.SetLastModel(modelName)
opts := runOptions{
Model: modelName,
WordWrap: os.Getenv("TERM") == "xterm-256color",
Options: map[string]any{},
ShowConnect: true,
}
if err := loadOrUnloadModel(cmd, &opts); err != nil {
fmt.Fprintf(os.Stderr, "Error loading model: %v\n", err)
return
}
if err := generateInteractive(cmd, opts); err != nil {
fmt.Fprintf(os.Stderr, "Error running model: %v\n", err)
}
}
type launcherDeps struct {
buildState func(context.Context) (*launch.LauncherState, error)
runMenu func(*launch.LauncherState) (tui.TUIAction, error)
resolveRunModel func(context.Context, launch.RunModelRequest) (string, error)
launchIntegration func(context.Context, launch.IntegrationLaunchRequest) error
runModel func(*cobra.Command, string) error
accountState func() *launch.AccountState
accountStateUpdates func(context.Context) <-chan *launch.AccountState
}
launchIntegration := func(name string) bool {
if err := config.EnsureInstalled(name); err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
return true
}
// If not configured or model no longer exists, prompt for model selection
configuredModel := config.IntegrationModel(name)
if configuredModel == "" || !config.ModelExists(cmd.Context(), configuredModel) || config.IsCloudModelDisabled(cmd.Context(), configuredModel) {
err := config.ConfigureIntegrationWithSelectors(cmd.Context(), name, singleSelector, multiSelector)
if errors.Is(err, config.ErrCancelled) {
return false // Return to main menu
}
if err != nil {
fmt.Fprintf(os.Stderr, "Error configuring %s: %v\n", name, err)
return true
}
}
if err := config.LaunchIntegration(name); err != nil {
fmt.Fprintf(os.Stderr, "Error launching %s: %v\n", name, err)
}
return true
}
func runInteractiveTUIStep(cmd *cobra.Command, deps launcherDeps) (bool, error) {
state, err := deps.buildState(cmd.Context())
if err != nil {
return false, fmt.Errorf("build launcher state: %w", err)
}
if state != nil && deps.accountState != nil {
state.AccountState = deps.accountState()
}
switch result.Selection {
case tui.SelectionNone:
// User quit
return
case tui.SelectionRunModel:
_ = config.SetLastSelection("run")
if modelName := config.LastModel(); modelName != "" && !config.IsCloudModelDisabled(cmd.Context(), modelName) {
runModel(modelName)
} else {
modelName, err := config.SelectModelWithSelector(cmd.Context(), singleSelector)
if errors.Is(err, config.ErrCancelled) {
continue // Return to main menu
}
if err != nil {
fmt.Fprintf(os.Stderr, "Error selecting model: %v\n", err)
continue
}
runModel(modelName)
}
case tui.SelectionChangeRunModel:
_ = config.SetLastSelection("run")
// Use model from modal if selected, otherwise show picker
modelName := result.Model
if modelName == "" {
var err error
modelName, err = config.SelectModelWithSelector(cmd.Context(), singleSelector)
if errors.Is(err, config.ErrCancelled) {
continue // Return to main menu
}
if err != nil {
fmt.Fprintf(os.Stderr, "Error selecting model: %v\n", err)
continue
}
}
if config.IsCloudModelDisabled(cmd.Context(), modelName) {
continue // Return to main menu
}
runModel(modelName)
case tui.SelectionIntegration:
_ = config.SetLastSelection(result.Integration)
if !launchIntegration(result.Integration) {
continue // Return to main menu
}
case tui.SelectionChangeIntegration:
_ = config.SetLastSelection(result.Integration)
if len(result.Models) > 0 {
// Filter out cloud-disabled models
var filtered []string
for _, m := range result.Models {
if !config.IsCloudModelDisabled(cmd.Context(), m) {
filtered = append(filtered, m)
}
}
if len(filtered) == 0 {
continue
}
result.Models = filtered
// Multi-select from modal (Editor integrations)
if err := config.SaveAndEditIntegration(result.Integration, result.Models); err != nil {
fmt.Fprintf(os.Stderr, "Error configuring %s: %v\n", result.Integration, err)
continue
}
if err := config.LaunchIntegrationWithModel(result.Integration, result.Models[0]); err != nil {
fmt.Fprintf(os.Stderr, "Error launching %s: %v\n", result.Integration, err)
}
} else if result.Model != "" {
if config.IsCloudModelDisabled(cmd.Context(), result.Model) {
continue
}
// Single-select from modal - save and launch
if err := config.SaveIntegration(result.Integration, []string{result.Model}); err != nil {
fmt.Fprintf(os.Stderr, "Error saving config: %v\n", err)
continue
}
if err := config.LaunchIntegrationWithModel(result.Integration, result.Model); err != nil {
fmt.Fprintf(os.Stderr, "Error launching %s: %v\n", result.Integration, err)
}
} else {
err := config.ConfigureIntegrationWithSelectors(cmd.Context(), result.Integration, singleSelector, multiSelector)
if errors.Is(err, config.ErrCancelled) {
continue // Return to main menu
}
if err != nil {
fmt.Fprintf(os.Stderr, "Error configuring %s: %v\n", result.Integration, err)
continue
}
if err := config.LaunchIntegration(result.Integration); err != nil {
fmt.Fprintf(os.Stderr, "Error launching %s: %v\n", result.Integration, err)
}
}
action, err := deps.runMenu(state)
if err != nil {
return false, fmt.Errorf("run launcher menu: %w", err)
}
return runLauncherAction(cmd, action, deps)
}
func saveLauncherSelection(action tui.TUIAction) {
// Best effort only: this affects menu recall, not launch correctness.
_ = config.SetLastSelection(action.LastSelection())
}
func runLauncherAction(cmd *cobra.Command, action tui.TUIAction, deps launcherDeps) (bool, error) {
switch action.Kind {
case tui.TUIActionNone:
return false, nil
case tui.TUIActionRunModel:
saveLauncherSelection(action)
req := action.RunModelRequest()
if deps.accountState != nil {
req.AccountState = deps.accountState()
req.AccountStateProvider = deps.accountState
}
req.AccountStateUpdates = deps.accountStateUpdates
modelName, err := deps.resolveRunModel(cmd.Context(), req)
if errors.Is(err, launch.ErrCancelled) {
return true, nil
}
if err != nil {
return true, fmt.Errorf("selecting model: %w", err)
}
if err := deps.runModel(cmd, modelName); err != nil {
return true, err
}
return true, nil
case tui.TUIActionLaunchIntegration:
saveLauncherSelection(action)
req := action.IntegrationLaunchRequest()
if deps.accountState != nil {
req.AccountState = deps.accountState()
req.AccountStateProvider = deps.accountState
}
req.AccountStateUpdates = deps.accountStateUpdates
err := deps.launchIntegration(cmd.Context(), req)
if errors.Is(err, launch.ErrCancelled) {
return true, nil
}
if err != nil {
return true, fmt.Errorf("launching %s: %w", action.Integration, err)
}
if launcherActionExitsLoop(action.Integration) {
return false, nil
}
return true, nil
default:
return false, fmt.Errorf("unknown launcher action: %d", action.Kind)
}
}
func launcherActionExitsLoop(integration string) bool {
switch integration {
case "claude-desktop", "vscode":
return true
default:
return false
}
}
@@ -2111,6 +2277,9 @@ func NewCLI() *cobra.Command {
if experimental, _ := cmd.Flags().GetBool("experimental"); experimental {
return nil
}
if draftQuantize, _ := cmd.Flags().GetString("draft-quantize"); draftQuantize != "" {
return errors.New("--draft-quantize requires --experimental")
}
return checkServerHeartbeat(cmd, args)
},
RunE: CreateHandler,
@@ -2118,6 +2287,7 @@ func NewCLI() *cobra.Command {
createCmd.Flags().StringP("file", "f", "", "Name of the Modelfile (default \"Modelfile\")")
createCmd.Flags().StringP("quantize", "q", "", "Quantize model to this level (e.g. q4_K_M)")
createCmd.Flags().String("draft-quantize", "", "Quantize draft model to this level")
createCmd.Flags().Bool("experimental", false, "Enable experimental safetensors model creation")
showCmd := &cobra.Command{
@@ -2338,7 +2508,7 @@ func NewCLI() *cobra.Command {
copyCmd,
deleteCmd,
runnerCmd,
config.LaunchCmd(checkServerHeartbeat, runInteractiveTUI),
launch.LaunchCmd(checkServerHeartbeat, runInteractiveTUI),
)
return rootCmd
+305
View File
@@ -0,0 +1,305 @@
package cmd
import (
"context"
"testing"
"github.com/spf13/cobra"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/launch"
"github.com/ollama/ollama/cmd/tui"
)
func setCmdTestHome(t *testing.T, dir string) {
t.Helper()
t.Setenv("HOME", dir)
t.Setenv("USERPROFILE", dir)
}
func unexpectedRunModelResolution(t *testing.T) func(context.Context, launch.RunModelRequest) (string, error) {
t.Helper()
return func(ctx context.Context, req launch.RunModelRequest) (string, error) {
t.Fatalf("did not expect run-model resolution: %+v", req)
return "", nil
}
}
func unexpectedIntegrationLaunch(t *testing.T) func(context.Context, launch.IntegrationLaunchRequest) error {
t.Helper()
return func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
t.Fatalf("did not expect integration launch: %+v", req)
return nil
}
}
func unexpectedModelLaunch(t *testing.T) func(*cobra.Command, string) error {
t.Helper()
return func(cmd *cobra.Command, model string) error {
t.Fatalf("did not expect chat launch: %s", model)
return nil
}
}
func TestRunInteractiveTUI_RunModelActionsUseResolveRunModel(t *testing.T) {
tests := []struct {
name string
action tui.TUIAction
wantForce bool
wantModel string
}{
{
name: "enter uses saved model flow",
action: tui.TUIAction{Kind: tui.TUIActionRunModel},
wantModel: "qwen3:8b",
},
{
name: "right forces picker",
action: tui.TUIAction{Kind: tui.TUIActionRunModel, ForceConfigure: true},
wantForce: true,
wantModel: "glm-5:cloud",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
var menuCalls int
runMenu := func(state *launch.LauncherState) (tui.TUIAction, error) {
menuCalls++
if menuCalls == 1 {
return tt.action, nil
}
return tui.TUIAction{Kind: tui.TUIActionNone}, nil
}
var gotReq launch.RunModelRequest
var launched string
prefetchedAccount := &launch.AccountState{}
accountUpdates := func(context.Context) <-chan *launch.AccountState { return nil }
deps := launcherDeps{
buildState: func(ctx context.Context) (*launch.LauncherState, error) {
return &launch.LauncherState{}, nil
},
runMenu: func(state *launch.LauncherState) (tui.TUIAction, error) {
if state.AccountState != prefetchedAccount {
t.Fatalf("prefetched account state was not piped to menu state")
}
return runMenu(state)
},
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
gotReq = req
return tt.wantModel, nil
},
launchIntegration: unexpectedIntegrationLaunch(t),
runModel: func(cmd *cobra.Command, model string) error {
launched = model
return nil
},
accountState: func() *launch.AccountState {
return prefetchedAccount
},
accountStateUpdates: accountUpdates,
}
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
for {
continueLoop, err := runInteractiveTUIStep(cmd, deps)
if err != nil {
t.Fatalf("unexpected step error: %v", err)
}
if !continueLoop {
break
}
}
if gotReq.ForcePicker != tt.wantForce {
t.Fatalf("expected ForcePicker=%v, got %v", tt.wantForce, gotReq.ForcePicker)
}
if gotReq.AccountState != prefetchedAccount {
t.Fatalf("expected prefetched account state to be passed to run model request")
}
if gotReq.AccountStateUpdates == nil {
t.Fatalf("expected account state updates to be passed to run model request")
}
if launched != tt.wantModel {
t.Fatalf("expected interactive launcher to run %q, got %q", tt.wantModel, launched)
}
if got := config.LastSelection(); got != "run" {
t.Fatalf("expected last selection to be run, got %q", got)
}
})
}
}
func TestRunInteractiveTUI_IntegrationActionsUseLaunchIntegration(t *testing.T) {
tests := []struct {
name string
action tui.TUIAction
wantForce bool
}{
{
name: "enter launches integration",
action: tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude"},
},
{
name: "right forces configure",
action: tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude", ForceConfigure: true},
wantForce: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
setCmdTestHome(t, t.TempDir())
var menuCalls int
runMenu := func(state *launch.LauncherState) (tui.TUIAction, error) {
menuCalls++
if menuCalls == 1 {
return tt.action, nil
}
return tui.TUIAction{Kind: tui.TUIActionNone}, nil
}
var gotReq launch.IntegrationLaunchRequest
prefetchedAccount := &launch.AccountState{}
accountUpdates := func(context.Context) <-chan *launch.AccountState { return nil }
deps := launcherDeps{
buildState: func(ctx context.Context) (*launch.LauncherState, error) {
return &launch.LauncherState{}, nil
},
runMenu: func(state *launch.LauncherState) (tui.TUIAction, error) {
if state.AccountState != prefetchedAccount {
t.Fatalf("prefetched account state was not piped to menu state")
}
return runMenu(state)
},
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
gotReq = req
return nil
},
runModel: unexpectedModelLaunch(t),
accountState: func() *launch.AccountState {
return prefetchedAccount
},
accountStateUpdates: accountUpdates,
}
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
for {
continueLoop, err := runInteractiveTUIStep(cmd, deps)
if err != nil {
t.Fatalf("unexpected step error: %v", err)
}
if !continueLoop {
break
}
}
if gotReq.Name != "claude" {
t.Fatalf("expected integration name to be passed through, got %q", gotReq.Name)
}
if gotReq.ForceConfigure != tt.wantForce {
t.Fatalf("expected ForceConfigure=%v, got %v", tt.wantForce, gotReq.ForceConfigure)
}
if gotReq.AccountState != prefetchedAccount {
t.Fatalf("expected prefetched account state to be passed to integration request")
}
if gotReq.AccountStateUpdates == nil {
t.Fatalf("expected account state updates to be passed to integration request")
}
if got := config.LastSelection(); got != "claude" {
t.Fatalf("expected last selection to be claude, got %q", got)
}
})
}
}
func TestRunLauncherAction_RunModelContinuesAfterCancellation(t *testing.T) {
setCmdTestHome(t, t.TempDir())
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionRunModel}, launcherDeps{
buildState: nil,
runMenu: nil,
resolveRunModel: func(ctx context.Context, req launch.RunModelRequest) (string, error) {
return "", launch.ErrCancelled
},
launchIntegration: unexpectedIntegrationLaunch(t),
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error on cancellation, got %v", err)
}
if !continueLoop {
t.Fatal("expected cancellation to continue the menu loop")
}
}
func TestRunLauncherAction_GUIAppsExitTUILoop(t *testing.T) {
setCmdTestHome(t, t.TempDir())
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
for _, integration := range []string{"claude-desktop", "vscode"} {
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: integration}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return nil
},
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error for %s, got %v", integration, err)
}
if continueLoop {
t.Fatalf("expected %s launch to exit the TUI loop (return false)", integration)
}
}
// Other integrations should continue the TUI loop (return true).
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude"}, launcherDeps{
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return nil
},
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
if !continueLoop {
t.Fatal("expected non-vscode integration to continue the TUI loop (return true)")
}
}
func TestRunLauncherAction_IntegrationContinuesAfterCancellation(t *testing.T) {
setCmdTestHome(t, t.TempDir())
cmd := &cobra.Command{}
cmd.SetContext(context.Background())
continueLoop, err := runLauncherAction(cmd, tui.TUIAction{Kind: tui.TUIActionLaunchIntegration, Integration: "claude"}, launcherDeps{
buildState: nil,
runMenu: nil,
resolveRunModel: unexpectedRunModelResolution(t),
launchIntegration: func(ctx context.Context, req launch.IntegrationLaunchRequest) error {
return launch.ErrCancelled
},
runModel: unexpectedModelLaunch(t),
})
if err != nil {
t.Fatalf("expected nil error on cancellation, got %v", err)
}
if !continueLoop {
t.Fatal("expected cancellation to continue the menu loop")
}
}
+633 -34
View File
@@ -9,6 +9,7 @@ import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@@ -301,7 +302,7 @@ Weigh anchor!
ParameterSize: "7B",
QuantizationLevel: "FP16",
},
Requires: "0.14.0",
Requires: "0.19.0",
}, false, &b); err != nil {
t.Fatal(err)
}
@@ -310,10 +311,17 @@ Weigh anchor!
architecture test
parameters 7B
quantization FP16
requires 0.14.0
requires 0.19.0
`
if diff := cmp.Diff(expect, b.String()); diff != "" {
trimLinePadding := func(s string) string {
lines := strings.Split(s, "\n")
for i, line := range lines {
lines[i] = strings.TrimRight(line, " \t\r")
}
return strings.Join(lines, "\n")
}
if diff := cmp.Diff(trimLinePadding(expect), trimLinePadding(b.String())); diff != "" {
t.Errorf("unexpected output (-want +got):\n%s", diff)
}
})
@@ -705,6 +713,347 @@ func TestRunEmbeddingModelNoInput(t *testing.T) {
}
}
func TestRunHandler_CloudAuthErrorOnShow_PrintsSigninMessage(t *testing.T) {
var generateCalled bool
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/show" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusUnauthorized)
if err := json.NewEncoder(w).Encode(map[string]string{
"error": "unauthorized",
"signin_url": "https://ollama.com/signin",
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/generate" && r.Method == http.MethodPost:
generateCalled = true
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.GenerateResponse{Done: true}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
default:
http.NotFound(w, r)
}
}))
t.Setenv("OLLAMA_HOST", mockServer.URL)
t.Cleanup(mockServer.Close)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
cmd.Flags().String("keepalive", "", "")
cmd.Flags().Bool("truncate", false, "")
cmd.Flags().Int("dimensions", 0, "")
cmd.Flags().Bool("verbose", false, "")
cmd.Flags().Bool("insecure", false, "")
cmd.Flags().Bool("nowordwrap", false, "")
cmd.Flags().String("format", "", "")
cmd.Flags().String("think", "", "")
cmd.Flags().Bool("hidethinking", false, "")
oldStdout := os.Stdout
readOut, writeOut, _ := os.Pipe()
os.Stdout = writeOut
t.Cleanup(func() { os.Stdout = oldStdout })
err := RunHandler(cmd, []string{"gpt-oss:20b:cloud", "hi"})
_ = writeOut.Close()
var out bytes.Buffer
_, _ = io.Copy(&out, readOut)
if err != nil {
t.Fatalf("RunHandler returned error: %v", err)
}
if generateCalled {
t.Fatal("expected run to stop before /api/generate after unauthorized /api/show")
}
if !strings.Contains(out.String(), "You need to be signed in to Ollama to run Cloud models.") {
t.Fatalf("expected sign-in guidance message, got %q", out.String())
}
if !strings.Contains(out.String(), "https://ollama.com/signin") {
t.Fatalf("expected signin_url in output, got %q", out.String())
}
}
func TestRunHandler_CloudAuthErrorOnGenerate_PrintsSigninMessage(t *testing.T) {
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/show" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []model.Capability{model.CapabilityCompletion},
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/generate" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusUnauthorized)
if err := json.NewEncoder(w).Encode(map[string]string{
"error": "unauthorized",
"signin_url": "https://ollama.com/signin",
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
default:
http.NotFound(w, r)
}
}))
t.Setenv("OLLAMA_HOST", mockServer.URL)
t.Cleanup(mockServer.Close)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
cmd.Flags().String("keepalive", "", "")
cmd.Flags().Bool("truncate", false, "")
cmd.Flags().Int("dimensions", 0, "")
cmd.Flags().Bool("verbose", false, "")
cmd.Flags().Bool("insecure", false, "")
cmd.Flags().Bool("nowordwrap", false, "")
cmd.Flags().String("format", "", "")
cmd.Flags().String("think", "", "")
cmd.Flags().Bool("hidethinking", false, "")
oldStdout := os.Stdout
readOut, writeOut, _ := os.Pipe()
os.Stdout = writeOut
t.Cleanup(func() { os.Stdout = oldStdout })
err := RunHandler(cmd, []string{"gpt-oss:20b:cloud", "hi"})
_ = writeOut.Close()
var out bytes.Buffer
_, _ = io.Copy(&out, readOut)
if err != nil {
t.Fatalf("RunHandler returned error: %v", err)
}
if !strings.Contains(out.String(), "You need to be signed in to Ollama to run Cloud models.") {
t.Fatalf("expected sign-in guidance message, got %q", out.String())
}
if !strings.Contains(out.String(), "https://ollama.com/signin") {
t.Fatalf("expected signin_url in output, got %q", out.String())
}
}
func TestRunHandler_ExplicitCloudStubMissing_PullsNormalizedNameTEMP(t *testing.T) {
var pulledModel string
var generateCalled bool
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/show" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []model.Capability{model.CapabilityCompletion},
RemoteModel: "gpt-oss:20b",
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/tags" && r.Method == http.MethodGet:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ListResponse{Models: nil}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/pull" && r.Method == http.MethodPost:
var req api.PullRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pulledModel = req.Model
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ProgressResponse{Status: "success"}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/generate" && r.Method == http.MethodPost:
generateCalled = true
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.GenerateResponse{Done: true}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
default:
http.NotFound(w, r)
}
}))
t.Setenv("OLLAMA_HOST", mockServer.URL)
t.Cleanup(mockServer.Close)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
cmd.Flags().String("keepalive", "", "")
cmd.Flags().Bool("truncate", false, "")
cmd.Flags().Int("dimensions", 0, "")
cmd.Flags().Bool("verbose", false, "")
cmd.Flags().Bool("insecure", false, "")
cmd.Flags().Bool("nowordwrap", false, "")
cmd.Flags().String("format", "", "")
cmd.Flags().String("think", "", "")
cmd.Flags().Bool("hidethinking", false, "")
err := RunHandler(cmd, []string{"gpt-oss:20b:cloud", "hi"})
if err != nil {
t.Fatalf("RunHandler returned error: %v", err)
}
if pulledModel != "gpt-oss:20b-cloud" {
t.Fatalf("expected normalized pull model %q, got %q", "gpt-oss:20b-cloud", pulledModel)
}
if !generateCalled {
t.Fatal("expected /api/generate to be called")
}
}
func TestRunHandler_ExplicitCloudStubPresent_SkipsPullTEMP(t *testing.T) {
var pullCalled bool
var generateCalled bool
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/show" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []model.Capability{model.CapabilityCompletion},
RemoteModel: "gpt-oss:20b",
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/tags" && r.Method == http.MethodGet:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ListResponse{
Models: []api.ListModelResponse{{Name: "gpt-oss:20b-cloud"}},
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/pull" && r.Method == http.MethodPost:
pullCalled = true
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ProgressResponse{Status: "success"}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/generate" && r.Method == http.MethodPost:
generateCalled = true
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.GenerateResponse{Done: true}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
default:
http.NotFound(w, r)
}
}))
t.Setenv("OLLAMA_HOST", mockServer.URL)
t.Cleanup(mockServer.Close)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
cmd.Flags().String("keepalive", "", "")
cmd.Flags().Bool("truncate", false, "")
cmd.Flags().Int("dimensions", 0, "")
cmd.Flags().Bool("verbose", false, "")
cmd.Flags().Bool("insecure", false, "")
cmd.Flags().Bool("nowordwrap", false, "")
cmd.Flags().String("format", "", "")
cmd.Flags().String("think", "", "")
cmd.Flags().Bool("hidethinking", false, "")
err := RunHandler(cmd, []string{"gpt-oss:20b:cloud", "hi"})
if err != nil {
t.Fatalf("RunHandler returned error: %v", err)
}
if pullCalled {
t.Fatal("expected /api/pull not to be called when cloud stub already exists")
}
if !generateCalled {
t.Fatal("expected /api/generate to be called")
}
}
func TestRunHandler_ExplicitCloudStubPullFailure_IsBestEffortTEMP(t *testing.T) {
var generateCalled bool
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.URL.Path == "/api/show" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ShowResponse{
Capabilities: []model.Capability{model.CapabilityCompletion},
RemoteModel: "gpt-oss:20b",
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/tags" && r.Method == http.MethodGet:
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.ListResponse{Models: nil}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/pull" && r.Method == http.MethodPost:
w.WriteHeader(http.StatusInternalServerError)
if err := json.NewEncoder(w).Encode(map[string]string{"error": "pull failed"}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
case r.URL.Path == "/api/generate" && r.Method == http.MethodPost:
generateCalled = true
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(api.GenerateResponse{Done: true}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
return
default:
http.NotFound(w, r)
}
}))
t.Setenv("OLLAMA_HOST", mockServer.URL)
t.Cleanup(mockServer.Close)
cmd := &cobra.Command{}
cmd.SetContext(t.Context())
cmd.Flags().String("keepalive", "", "")
cmd.Flags().Bool("truncate", false, "")
cmd.Flags().Int("dimensions", 0, "")
cmd.Flags().Bool("verbose", false, "")
cmd.Flags().Bool("insecure", false, "")
cmd.Flags().Bool("nowordwrap", false, "")
cmd.Flags().String("format", "", "")
cmd.Flags().String("think", "", "")
cmd.Flags().Bool("hidethinking", false, "")
err := RunHandler(cmd, []string{"gpt-oss:20b:cloud", "hi"})
if err != nil {
t.Fatalf("RunHandler returned error: %v", err)
}
if !generateCalled {
t.Fatal("expected /api/generate to be called despite pull failure")
}
}
func TestGetModelfileName(t *testing.T) {
tests := []struct {
name string
@@ -1176,6 +1525,87 @@ func TestCreateHandler(t *testing.T) {
}
}
func TestCreateHandlerDraftQuantizeRequiresExperimental(t *testing.T) {
cmd := &cobra.Command{}
cmd.Flags().Bool("experimental", false, "")
cmd.Flags().String("draft-quantize", "mxfp8", "")
cmd.SetContext(t.Context())
err := CreateHandler(cmd, []string{"test-model"})
if err == nil || !strings.Contains(err.Error(), "--draft-quantize requires --experimental") {
t.Fatalf("error = %v, want draft-quantize requires experimental", err)
}
}
func TestCreateHandlerDraftRequiresExperimental(t *testing.T) {
dir := t.TempDir()
modelfile := filepath.Join(dir, "Modelfile")
if err := os.WriteFile(modelfile, []byte("FROM base\nDRAFT ./assistant\n"), 0o644); err != nil {
t.Fatal(err)
}
cmd := &cobra.Command{}
cmd.Flags().Bool("experimental", false, "")
cmd.Flags().String("draft-quantize", "", "")
cmd.Flags().String("file", modelfile, "")
cmd.SetContext(t.Context())
err := CreateHandler(cmd, []string{"test-model"})
if err == nil || !strings.Contains(err.Error(), "DRAFT requires --experimental") {
t.Fatalf("error = %v, want DRAFT requires --experimental", err)
}
}
func TestResolveExperimentalLocalModelDir(t *testing.T) {
dir := t.TempDir()
modelfile := filepath.Join(dir, "Modelfile")
modelDir := filepath.Join(dir, "model")
if err := os.Mkdir(modelDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(modelDir, "config.json"), []byte(`{}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(modelDir, "model.safetensors"), []byte("dummy"), 0o644); err != nil {
t.Fatal(err)
}
if got := resolveExperimentalLocalModelDir("gemma4", modelfile); got != "gemma4" {
t.Fatalf("resolveExperimentalLocalModelDir(model name) = %q, want gemma4", got)
}
if got := resolveExperimentalLocalModelDir("./model", modelfile); got != modelDir {
t.Fatalf("resolveExperimentalLocalModelDir(local dir) = %q, want %q", got, modelDir)
}
}
func TestResolveExperimentalDraftDir(t *testing.T) {
dir := t.TempDir()
modelfile := filepath.Join(dir, "Modelfile")
draftDir := filepath.Join(dir, "assistant")
if err := os.Mkdir(draftDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(draftDir, "config.json"), []byte(`{}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(draftDir, "model.safetensors"), []byte("dummy"), 0o644); err != nil {
t.Fatal(err)
}
got, err := resolveExperimentalDraftDir("./assistant", modelfile)
if err != nil {
t.Fatal(err)
}
if got != draftDir {
t.Fatalf("resolveExperimentalDraftDir(local dir) = %q, want %q", got, draftDir)
}
_, err = resolveExperimentalDraftDir("assistant-model", modelfile)
if err == nil || !strings.Contains(err.Error(), "DRAFT model references are not supported with --experimental yet") {
t.Fatalf("error = %v, want unsupported draft model reference", err)
}
}
func TestNewCreateRequest(t *testing.T) {
tests := []struct {
name string
@@ -1212,6 +1642,20 @@ func TestNewCreateRequest(t *testing.T) {
Model: "newmodel",
},
},
{
"explicit cloud model preserves source when parent lacks it",
"newmodel",
runOptions{
Model: "qwen3.5:cloud",
ParentModel: "qwen3.5",
Messages: []api.Message{},
WordWrap: true,
},
&api.CreateRequest{
From: "qwen3.5:cloud",
Model: "newmodel",
},
},
{
"parent model as filepath test",
"newmodel",
@@ -1293,6 +1737,24 @@ func TestNewCreateRequest(t *testing.T) {
},
},
},
{
"loaded messages are preserved when saving",
"newmodel",
runOptions{
Model: "mymodel",
ParentModel: "parentmodel",
LoadedMessages: []api.Message{{Role: "assistant", Content: "loaded"}},
Messages: []api.Message{{Role: "user", Content: "new"}},
},
&api.CreateRequest{
From: "parentmodel",
Model: "newmodel",
Messages: []api.Message{
{Role: "assistant", Content: "loaded"},
{Role: "user", Content: "new"},
},
},
},
}
for _, tt := range tests {
@@ -1305,15 +1767,43 @@ func TestNewCreateRequest(t *testing.T) {
}
}
func TestApplyShowResponseToRunOptions(t *testing.T) {
opts := runOptions{}
info := &api.ShowResponse{
Details: api.ModelDetails{
ParentModel: "parentmodel",
},
Messages: []api.Message{
{Role: "assistant", Content: "loaded"},
},
}
applyShowResponseToRunOptions(&opts, info)
if opts.ParentModel != "parentmodel" {
t.Fatalf("ParentModel = %q, want %q", opts.ParentModel, "parentmodel")
}
if !cmp.Equal(opts.LoadedMessages, info.Messages) {
t.Fatalf("LoadedMessages = %#v, want %#v", opts.LoadedMessages, info.Messages)
}
info.Messages[0].Content = "modified"
if opts.LoadedMessages[0].Content == "modified" {
t.Fatal("LoadedMessages should be copied independently from ShowResponse")
}
}
func TestRunOptions_Copy(t *testing.T) {
// Setup test data
originalKeepAlive := &api.Duration{Duration: 5 * time.Minute}
originalThink := &api.ThinkValue{Value: "test reasoning"}
original := runOptions{
Model: "test-model",
ParentModel: "parent-model",
Prompt: "test prompt",
Model: "test-model",
ParentModel: "parent-model",
LoadedMessages: []api.Message{{Role: "assistant", Content: "loaded hello"}},
Prompt: "test prompt",
Messages: []api.Message{
{Role: "user", Content: "hello"},
{Role: "assistant", Content: "hi there"},
@@ -1353,6 +1843,7 @@ func TestRunOptions_Copy(t *testing.T) {
}{
{"Model", copied.Model, original.Model},
{"ParentModel", copied.ParentModel, original.ParentModel},
{"LoadedMessages", copied.LoadedMessages, original.LoadedMessages},
{"Prompt", copied.Prompt, original.Prompt},
{"WordWrap", copied.WordWrap, original.WordWrap},
{"Format", copied.Format, original.Format},
@@ -1457,13 +1948,18 @@ func TestRunOptions_Copy(t *testing.T) {
func TestRunOptions_Copy_EmptySlicesAndMaps(t *testing.T) {
// Test with empty slices and maps
original := runOptions{
Messages: []api.Message{},
Images: []api.ImageData{},
Options: map[string]any{},
LoadedMessages: []api.Message{},
Messages: []api.Message{},
Images: []api.ImageData{},
Options: map[string]any{},
}
copied := original.Copy()
if copied.LoadedMessages == nil {
t.Error("Empty LoadedMessages slice should remain empty, not nil")
}
if copied.Messages == nil {
t.Error("Empty Messages slice should remain empty, not nil")
}
@@ -1480,6 +1976,10 @@ func TestRunOptions_Copy_EmptySlicesAndMaps(t *testing.T) {
t.Error("Empty Messages slice should remain empty")
}
if len(copied.LoadedMessages) != 0 {
t.Error("Empty LoadedMessages slice should remain empty")
}
if len(copied.Images) != 0 {
t.Error("Empty Images slice should remain empty")
}
@@ -1557,7 +2057,7 @@ func TestShowInfoImageGen(t *testing.T) {
QuantizationLevel: "Q8",
},
Capabilities: []model.Capability{model.CapabilityImage},
Requires: "0.14.0",
Requires: "0.19.0",
}, false, &b)
if err != nil {
t.Fatal(err)
@@ -1567,7 +2067,7 @@ func TestShowInfoImageGen(t *testing.T) {
" architecture ZImagePipeline \n" +
" parameters 10.3B \n" +
" quantization Q8 \n" +
" requires 0.14.0 \n" +
" requires 0.19.0 \n" +
"\n" +
" Capabilities\n" +
" image \n" +
@@ -1625,16 +2125,20 @@ func TestRunOptions_Copy_Independence(t *testing.T) {
// Test that modifications to original don't affect copy
originalThink := &api.ThinkValue{Value: "original"}
original := runOptions{
Model: "original-model",
Messages: []api.Message{{Role: "user", Content: "original"}},
Options: map[string]any{"key": "value"},
Think: originalThink,
Model: "original-model",
LoadedMessages: []api.Message{{Role: "assistant", Content: "loaded"}},
Messages: []api.Message{{Role: "user", Content: "original"}},
Options: map[string]any{"key": "value"},
Think: originalThink,
}
copied := original.Copy()
// Modify original
original.Model = "modified-model"
if len(original.LoadedMessages) > 0 {
original.LoadedMessages[0].Content = "modified loaded"
}
if len(original.Messages) > 0 {
original.Messages[0].Content = "modified"
}
@@ -1648,6 +2152,10 @@ func TestRunOptions_Copy_Independence(t *testing.T) {
t.Error("Copy Model should not be affected by original modification")
}
if len(copied.LoadedMessages) > 0 && copied.LoadedMessages[0].Content == "modified loaded" {
t.Error("Copy LoadedMessages should not be affected by original modification")
}
if len(copied.Messages) > 0 && copied.Messages[0].Content == "modified" {
t.Error("Copy Messages should not be affected by original modification")
}
@@ -1663,31 +2171,81 @@ func TestRunOptions_Copy_Independence(t *testing.T) {
func TestLoadOrUnloadModel_CloudModelAuth(t *testing.T) {
tests := []struct {
name string
remoteHost string
whoamiStatus int
whoamiResp any
expectedError string
name string
model string
showStatus int
remoteHost string
remoteModel string
whoamiStatus int
whoamiResp any
expectWhoami bool
expectedError string
expectAuthError bool
}{
{
name: "ollama.com cloud model - user signed in",
model: "test-cloud-model",
remoteHost: "https://ollama.com",
remoteModel: "test-model",
whoamiStatus: http.StatusOK,
whoamiResp: api.UserResponse{Name: "testuser"},
expectWhoami: true,
},
{
name: "ollama.com cloud model - user not signed in",
model: "test-cloud-model",
remoteHost: "https://ollama.com",
remoteModel: "test-model",
whoamiStatus: http.StatusUnauthorized,
whoamiResp: map[string]string{
"error": "unauthorized",
"signin_url": "https://ollama.com/signin",
},
expectedError: "unauthorized",
expectWhoami: true,
expectedError: "unauthorized",
expectAuthError: true,
},
{
name: "non-ollama.com remote - no auth check",
model: "test-cloud-model",
remoteHost: "https://other-remote.com",
remoteModel: "test-model",
whoamiStatus: http.StatusUnauthorized, // should not be called
whoamiResp: nil,
},
{
name: "explicit :cloud model - auth check without remote metadata",
model: "kimi-k2.5:cloud",
remoteHost: "",
remoteModel: "",
whoamiStatus: http.StatusOK,
whoamiResp: api.UserResponse{Name: "testuser"},
expectWhoami: true,
},
{
name: "explicit :cloud model without local stub returns not found by default",
model: "minimax-m2.7:cloud",
showStatus: http.StatusNotFound,
whoamiStatus: http.StatusOK,
whoamiResp: api.UserResponse{Name: "testuser"},
expectedError: "not found",
expectWhoami: false,
expectAuthError: false,
},
{
name: "explicit -cloud model - auth check without remote metadata",
model: "kimi-k2.5:latest-cloud",
remoteHost: "",
remoteModel: "",
whoamiStatus: http.StatusOK,
whoamiResp: api.UserResponse{Name: "testuser"},
expectWhoami: true,
},
{
name: "dash cloud-like name without explicit source does not require auth",
model: "test-cloud-model",
remoteHost: "",
remoteModel: "",
whoamiStatus: http.StatusUnauthorized, // should not be called
whoamiResp: nil,
},
@@ -1699,10 +2257,15 @@ func TestLoadOrUnloadModel_CloudModelAuth(t *testing.T) {
mockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
if tt.showStatus != 0 && tt.showStatus != http.StatusOK {
w.WriteHeader(tt.showStatus)
_ = json.NewEncoder(w).Encode(map[string]string{"error": "not found"})
return
}
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(api.ShowResponse{
RemoteHost: tt.remoteHost,
RemoteModel: "test-model",
RemoteModel: tt.remoteModel,
}); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
@@ -1715,6 +2278,8 @@ func TestLoadOrUnloadModel_CloudModelAuth(t *testing.T) {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
case "/api/generate":
w.WriteHeader(http.StatusOK)
default:
http.NotFound(w, r)
}
@@ -1727,29 +2292,28 @@ func TestLoadOrUnloadModel_CloudModelAuth(t *testing.T) {
cmd.SetContext(t.Context())
opts := &runOptions{
Model: "test-cloud-model",
Model: tt.model,
ShowConnect: false,
}
err := loadOrUnloadModel(cmd, opts)
if strings.HasPrefix(tt.remoteHost, "https://ollama.com") {
if !whoamiCalled {
t.Error("expected whoami to be called for ollama.com cloud model")
}
} else {
if whoamiCalled {
t.Error("whoami should not be called for non-ollama.com remote")
}
if whoamiCalled != tt.expectWhoami {
t.Errorf("whoami called = %v, want %v", whoamiCalled, tt.expectWhoami)
}
if tt.expectedError != "" {
if err == nil {
t.Errorf("expected error containing %q, got nil", tt.expectedError)
} else {
var authErr api.AuthorizationError
if !errors.As(err, &authErr) {
t.Errorf("expected AuthorizationError, got %T: %v", err, err)
if !tt.expectAuthError && !strings.Contains(strings.ToLower(err.Error()), strings.ToLower(tt.expectedError)) {
t.Errorf("expected error containing %q, got %v", tt.expectedError, err)
}
if tt.expectAuthError {
var authErr api.AuthorizationError
if !errors.As(err, &authErr) {
t.Errorf("expected AuthorizationError, got %T: %v", err, err)
}
}
}
} else {
@@ -1760,3 +2324,38 @@ func TestLoadOrUnloadModel_CloudModelAuth(t *testing.T) {
})
}
}
func TestIsLocalhost(t *testing.T) {
tests := []struct {
name string
host string
expected bool
}{
{"default empty", "", true},
{"localhost no port", "localhost", true},
{"localhost with port", "localhost:11435", true},
{"127.0.0.1 no port", "127.0.0.1", true},
{"127.0.0.1 with port", "127.0.0.1:11434", true},
{"0.0.0.0 no port", "0.0.0.0", true},
{"0.0.0.0 with port", "0.0.0.0:11434", true},
{"::1 no port", "::1", true},
{"[::1] with port", "[::1]:11434", true},
{"loopback with scheme", "http://localhost:11434", true},
{"remote hostname", "example.com", false},
{"remote hostname with port", "example.com:11434", false},
{"remote IP", "192.168.1.1", false},
{"remote IP with port", "192.168.1.1:11434", false},
{"remote with scheme", "http://example.com:11434", false},
{"https remote", "https://example.com:443", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv("OLLAMA_HOST", tt.host)
got := isLocalhost()
if got != tt.expected {
t.Errorf("isLocalhost() with OLLAMA_HOST=%q = %v, want %v", tt.host, got, tt.expected)
}
})
}
}
-192
View File
@@ -1,192 +0,0 @@
package config
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
)
// Claude implements Runner and AliasConfigurer for Claude Code integration
type Claude struct{}
// Compile-time check that Claude implements AliasConfigurer
var _ AliasConfigurer = (*Claude)(nil)
func (c *Claude) String() string { return "Claude Code" }
func (c *Claude) args(model string, extra []string) []string {
var args []string
if model != "" {
args = append(args, "--model", model)
}
args = append(args, extra...)
return args
}
func (c *Claude) findPath() (string, error) {
if p, err := exec.LookPath("claude"); err == nil {
return p, nil
}
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
name := "claude"
if runtime.GOOS == "windows" {
name = "claude.exe"
}
fallback := filepath.Join(home, ".claude", "local", name)
if _, err := os.Stat(fallback); err != nil {
return "", err
}
return fallback, nil
}
func (c *Claude) Run(model string, args []string) error {
claudePath, err := c.findPath()
if err != nil {
return fmt.Errorf("claude is not installed, install from https://code.claude.com/docs/en/quickstart")
}
cmd := exec.Command(claudePath, c.args(model, args)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
env := append(os.Environ(),
"ANTHROPIC_BASE_URL="+envconfig.Host().String(),
"ANTHROPIC_API_KEY=",
"ANTHROPIC_AUTH_TOKEN=ollama",
)
env = append(env, c.modelEnvVars(model)...)
cmd.Env = env
return cmd.Run()
}
// modelEnvVars returns Claude Code env vars that route all model tiers through Ollama.
func (c *Claude) modelEnvVars(model string) []string {
primary := model
fast := model
if cfg, err := loadIntegration("claude"); err == nil && cfg.Aliases != nil {
if p := cfg.Aliases["primary"]; p != "" {
primary = p
}
if f := cfg.Aliases["fast"]; f != "" {
fast = f
}
}
return []string{
"ANTHROPIC_DEFAULT_OPUS_MODEL=" + primary,
"ANTHROPIC_DEFAULT_SONNET_MODEL=" + primary,
"ANTHROPIC_DEFAULT_HAIKU_MODEL=" + fast,
"CLAUDE_CODE_SUBAGENT_MODEL=" + primary,
}
}
// ConfigureAliases sets up model aliases for Claude Code.
// model: the model to use (if empty, user will be prompted to select)
// aliases: existing alias configuration to preserve/update
// Cloud-only: subagent routing (fast model) is gated to cloud models only until
// there is a better strategy for prompt caching on local models.
func (c *Claude) ConfigureAliases(ctx context.Context, model string, existingAliases map[string]string, force bool) (map[string]string, bool, error) {
aliases := make(map[string]string)
for k, v := range existingAliases {
aliases[k] = v
}
if model != "" {
aliases["primary"] = model
}
if !force && aliases["primary"] != "" {
client, _ := api.ClientFromEnvironment()
if isCloudModel(ctx, client, aliases["primary"]) {
if isCloudModel(ctx, client, aliases["fast"]) {
return aliases, false, nil
}
} else {
delete(aliases, "fast")
return aliases, false, nil
}
}
items, existingModels, cloudModels, client, err := listModels(ctx)
if err != nil {
return nil, false, err
}
fmt.Fprintf(os.Stderr, "\n%sModel Configuration%s\n\n", ansiBold, ansiReset)
if aliases["primary"] == "" || force {
primary, err := DefaultSingleSelector("Select model:", items, aliases["primary"])
if err != nil {
return nil, false, err
}
if err := pullIfNeeded(ctx, client, existingModels, primary); err != nil {
return nil, false, err
}
if err := ensureAuth(ctx, client, cloudModels, []string{primary}); err != nil {
return nil, false, err
}
aliases["primary"] = primary
}
if isCloudModel(ctx, client, aliases["primary"]) {
if aliases["fast"] == "" || !isCloudModel(ctx, client, aliases["fast"]) {
aliases["fast"] = aliases["primary"]
}
} else {
delete(aliases, "fast")
}
return aliases, true, nil
}
// SetAliases syncs the configured aliases to the Ollama server using prefix matching.
// Cloud-only: for local models (fast is empty), we delete any existing aliases to
// prevent stale routing to a previous cloud model.
func (c *Claude) SetAliases(ctx context.Context, aliases map[string]string) error {
client, err := api.ClientFromEnvironment()
if err != nil {
return err
}
prefixes := []string{"claude-sonnet-", "claude-haiku-"}
if aliases["fast"] == "" {
for _, prefix := range prefixes {
_ = client.DeleteAliasExperimental(ctx, &api.AliasDeleteRequest{Alias: prefix})
}
return nil
}
prefixAliases := map[string]string{
"claude-sonnet-": aliases["primary"],
"claude-haiku-": aliases["fast"],
}
var errs []string
for prefix, target := range prefixAliases {
req := &api.AliasRequest{
Alias: prefix,
Target: target,
PrefixMatching: true,
}
if err := client.SetAliasExperimental(ctx, req); err != nil {
errs = append(errs, prefix)
}
}
if len(errs) > 0 {
return fmt.Errorf("failed to set aliases: %v", errs)
}
return nil
}
-67
View File
@@ -1,67 +0,0 @@
package config
import (
"fmt"
"os"
"os/exec"
"strings"
"github.com/ollama/ollama/envconfig"
"golang.org/x/mod/semver"
)
// Codex implements Runner for Codex integration
type Codex struct{}
func (c *Codex) String() string { return "Codex" }
func (c *Codex) args(model string, extra []string) []string {
args := []string{"--oss"}
if model != "" {
args = append(args, "-m", model)
}
args = append(args, extra...)
return args
}
func (c *Codex) Run(model string, args []string) error {
if err := checkCodexVersion(); err != nil {
return err
}
cmd := exec.Command("codex", c.args(model, args)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Env = append(os.Environ(),
"OPENAI_BASE_URL="+envconfig.Host().String()+"/v1/",
"OPENAI_API_KEY=ollama",
)
return cmd.Run()
}
func checkCodexVersion() error {
if _, err := exec.LookPath("codex"); err != nil {
return fmt.Errorf("codex is not installed, install with: npm install -g @openai/codex")
}
out, err := exec.Command("codex", "--version").Output()
if err != nil {
return fmt.Errorf("failed to get codex version: %w", err)
}
// Parse output like "codex-cli 0.87.0"
fields := strings.Fields(strings.TrimSpace(string(out)))
if len(fields) < 2 {
return fmt.Errorf("unexpected codex version output: %s", string(out))
}
version := "v" + fields[len(fields)-1]
minVersion := "v0.81.0"
if semver.Compare(version, minVersion) < 0 {
return fmt.Errorf("codex version %s is too old, minimum required is %s, update with: npm update -g @openai/codex", fields[len(fields)-1], "0.81.0")
}
return nil
}
-31
View File
@@ -1,31 +0,0 @@
package config
import (
"slices"
"testing"
)
func TestCodexArgs(t *testing.T) {
c := &Codex{}
tests := []struct {
name string
model string
args []string
want []string
}{
{"with model", "llama3.2", nil, []string{"--oss", "-m", "llama3.2"}},
{"empty model", "", nil, []string{"--oss"}},
{"with model and profile", "qwen3-coder", []string{"-p", "myprofile"}, []string{"--oss", "-m", "qwen3-coder", "-p", "myprofile"}},
{"with sandbox flag", "llama3.2", []string{"--sandbox", "workspace-write"}, []string{"--oss", "-m", "llama3.2", "--sandbox", "workspace-write"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := c.args(tt.model, tt.args)
if !slices.Equal(got, tt.want) {
t.Errorf("args(%q, %v) = %v, want %v", tt.model, tt.args, got, tt.want)
}
})
}
}
+13 -30
View File
@@ -3,7 +3,6 @@
package config
import (
"context"
"encoding/json"
"errors"
"fmt"
@@ -11,7 +10,7 @@ import (
"path/filepath"
"strings"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/cmd/internal/fileutil"
)
type integration struct {
@@ -20,6 +19,9 @@ type integration struct {
Onboarded bool `json:"onboarded,omitempty"`
}
// IntegrationConfig is the persisted config for one integration.
type IntegrationConfig = integration
type config struct {
Integrations map[string]*integration `json:"integrations"`
LastModel string `json:"last_model,omitempty"`
@@ -124,7 +126,7 @@ func save(cfg *config) error {
return err
}
return writeWithBackup(path, data)
return fileutil.WriteWithBackup(path, data)
}
func SaveIntegration(appName string, models []string) error {
@@ -155,8 +157,8 @@ func SaveIntegration(appName string, models []string) error {
return save(cfg)
}
// integrationOnboarded marks an integration as onboarded in ollama's config.
func integrationOnboarded(appName string) error {
// MarkIntegrationOnboarded marks an integration as onboarded in Ollama's config.
func MarkIntegrationOnboarded(appName string) error {
cfg, err := load()
if err != nil {
return err
@@ -174,7 +176,7 @@ func integrationOnboarded(appName string) error {
// IntegrationModel returns the first configured model for an integration, or empty string if not configured.
func IntegrationModel(appName string) string {
integrationConfig, err := loadIntegration(appName)
integrationConfig, err := LoadIntegration(appName)
if err != nil || len(integrationConfig.Models) == 0 {
return ""
}
@@ -183,7 +185,7 @@ func IntegrationModel(appName string) string {
// IntegrationModels returns all configured models for an integration, or nil.
func IntegrationModels(appName string) []string {
integrationConfig, err := loadIntegration(appName)
integrationConfig, err := LoadIntegration(appName)
if err != nil || len(integrationConfig.Models) == 0 {
return nil
}
@@ -228,28 +230,8 @@ func SetLastSelection(selection string) error {
return save(cfg)
}
// ModelExists checks if a model exists on the Ollama server.
func ModelExists(ctx context.Context, name string) bool {
if name == "" {
return false
}
client, err := api.ClientFromEnvironment()
if err != nil {
return false
}
models, err := client.List(ctx)
if err != nil {
return false
}
for _, m := range models.Models {
if m.Name == name || strings.HasPrefix(m.Name, name+":") {
return true
}
}
return false
}
func loadIntegration(appName string) (*integration, error) {
// LoadIntegration returns the saved config for one integration.
func LoadIntegration(appName string) (*integration, error) {
cfg, err := load()
if err != nil {
return nil, err
@@ -263,7 +245,8 @@ func loadIntegration(appName string) (*integration, error) {
return integrationConfig, nil
}
func saveAliases(appName string, aliases map[string]string) error {
// SaveAliases replaces the saved aliases for one integration.
func SaveAliases(appName string, aliases map[string]string) error {
if appName == "" {
return errors.New("app name cannot be empty")
}
+41 -77
View File
@@ -1,7 +1,6 @@
package config
import (
"context"
"errors"
"os"
"path/filepath"
@@ -45,12 +44,12 @@ func TestSaveAliases_ReplacesNotMerges(t *testing.T) {
"primary": "cloud-model",
"fast": "cloud-model",
}
if err := saveAliases("claude", initial); err != nil {
if err := SaveAliases("claude", initial); err != nil {
t.Fatalf("failed to save initial aliases: %v", err)
}
// Verify both are saved
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -63,12 +62,12 @@ func TestSaveAliases_ReplacesNotMerges(t *testing.T) {
"primary": "local-model",
// fast intentionally missing
}
if err := saveAliases("claude", updated); err != nil {
if err := SaveAliases("claude", updated); err != nil {
t.Fatalf("failed to save updated aliases: %v", err)
}
// Verify fast is GONE (not merged/preserved)
loaded, err = loadIntegration("claude")
loaded, err = LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load after update: %v", err)
}
@@ -91,12 +90,12 @@ func TestSaveAliases_PreservesModels(t *testing.T) {
// Then update aliases
aliases := map[string]string{"primary": "new-model"}
if err := saveAliases("claude", aliases); err != nil {
if err := SaveAliases("claude", aliases); err != nil {
t.Fatalf("failed to save aliases: %v", err)
}
// Verify models are preserved
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -111,16 +110,16 @@ func TestSaveAliases_EmptyMap(t *testing.T) {
setTestHome(t, tmpDir)
// Save with aliases
if err := saveAliases("claude", map[string]string{"primary": "model", "fast": "model"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model", "fast": "model"}); err != nil {
t.Fatalf("failed to save: %v", err)
}
// Save empty map
if err := saveAliases("claude", map[string]string{}); err != nil {
if err := SaveAliases("claude", map[string]string{}); err != nil {
t.Fatalf("failed to save empty: %v", err)
}
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -135,16 +134,16 @@ func TestSaveAliases_NilMap(t *testing.T) {
setTestHome(t, tmpDir)
// Save with aliases first
if err := saveAliases("claude", map[string]string{"primary": "model"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model"}); err != nil {
t.Fatalf("failed to save: %v", err)
}
// Save nil map - should clear aliases
if err := saveAliases("claude", nil); err != nil {
if err := SaveAliases("claude", nil); err != nil {
t.Fatalf("failed to save nil: %v", err)
}
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -155,7 +154,7 @@ func TestSaveAliases_NilMap(t *testing.T) {
// TestSaveAliases_EmptyAppName returns error
func TestSaveAliases_EmptyAppName(t *testing.T) {
err := saveAliases("", map[string]string{"primary": "model"})
err := SaveAliases("", map[string]string{"primary": "model"})
if err == nil {
t.Error("expected error for empty app name")
}
@@ -165,12 +164,12 @@ func TestSaveAliases_CaseInsensitive(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if err := saveAliases("Claude", map[string]string{"primary": "model1"}); err != nil {
if err := SaveAliases("Claude", map[string]string{"primary": "model1"}); err != nil {
t.Fatalf("failed to save: %v", err)
}
// Load with different case
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -179,11 +178,11 @@ func TestSaveAliases_CaseInsensitive(t *testing.T) {
}
// Update with different case
if err := saveAliases("CLAUDE", map[string]string{"primary": "model2"}); err != nil {
if err := SaveAliases("CLAUDE", map[string]string{"primary": "model2"}); err != nil {
t.Fatalf("failed to update: %v", err)
}
loaded, err = loadIntegration("claude")
loaded, err = LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load after update: %v", err)
}
@@ -198,11 +197,11 @@ func TestSaveAliases_CreatesIntegration(t *testing.T) {
setTestHome(t, tmpDir)
// Save aliases for non-existent integration
if err := saveAliases("newintegration", map[string]string{"primary": "model"}); err != nil {
if err := SaveAliases("newintegration", map[string]string{"primary": "model"}); err != nil {
t.Fatalf("failed to save: %v", err)
}
loaded, err := loadIntegration("newintegration")
loaded, err := LoadIntegration("newintegration")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -371,12 +370,12 @@ func TestAtomicUpdate_ServerSucceedsConfigSaved(t *testing.T) {
t.Fatal("server should succeed")
}
if err := saveAliases("claude", map[string]string{"primary": "model"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model"}); err != nil {
t.Fatalf("saveAliases failed: %v", err)
}
// Verify it was actually saved
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -408,7 +407,7 @@ func TestConfigFile_PreservesUnknownFields(t *testing.T) {
os.WriteFile(configPath, []byte(initialConfig), 0o644)
// Update aliases
if err := saveAliases("claude", map[string]string{"primary": "model2"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model2"}); err != nil {
t.Fatalf("failed to save: %v", err)
}
@@ -440,11 +439,6 @@ func containsHelper(s, substr string) bool {
return false
}
func TestClaudeImplementsAliasConfigurer(t *testing.T) {
c := &Claude{}
var _ AliasConfigurer = c // Compile-time check
}
func TestModelNameEdgeCases(t *testing.T) {
testCases := []struct {
name string
@@ -464,11 +458,11 @@ func TestModelNameEdgeCases(t *testing.T) {
setTestHome(t, tmpDir)
aliases := map[string]string{"primary": tc.model}
if err := saveAliases("claude", aliases); err != nil {
if err := SaveAliases("claude", aliases); err != nil {
t.Fatalf("failed to save model %q: %v", tc.model, err)
}
loaded, err := loadIntegration("claude")
loaded, err := LoadIntegration("claude")
if err != nil {
t.Fatalf("failed to load: %v", err)
}
@@ -485,7 +479,7 @@ func TestSwitchingScenarios(t *testing.T) {
setTestHome(t, tmpDir)
// Initial cloud config
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "cloud-model",
"fast": "cloud-model",
}); err != nil {
@@ -493,13 +487,13 @@ func TestSwitchingScenarios(t *testing.T) {
}
// Switch to local (no fast)
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "local-model",
}); err != nil {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
if loaded.Aliases["fast"] != "" {
t.Errorf("fast should be removed, got %q", loaded.Aliases["fast"])
}
@@ -513,21 +507,21 @@ func TestSwitchingScenarios(t *testing.T) {
setTestHome(t, tmpDir)
// Initial local config
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "local-model",
}); err != nil {
t.Fatal(err)
}
// Switch to cloud (with fast)
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "cloud-model",
"fast": "cloud-model",
}); err != nil {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
if loaded.Aliases["fast"] != "cloud-model" {
t.Errorf("fast should be cloud-model, got %q", loaded.Aliases["fast"])
}
@@ -538,7 +532,7 @@ func TestSwitchingScenarios(t *testing.T) {
setTestHome(t, tmpDir)
// Initial cloud config
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "cloud-model-1",
"fast": "cloud-model-1",
}); err != nil {
@@ -546,14 +540,14 @@ func TestSwitchingScenarios(t *testing.T) {
}
// Switch to different cloud
if err := saveAliases("claude", map[string]string{
if err := SaveAliases("claude", map[string]string{
"primary": "cloud-model-2",
"fast": "cloud-model-2",
}); err != nil {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
if loaded.Aliases["primary"] != "cloud-model-2" {
t.Errorf("primary should be cloud-model-2, got %q", loaded.Aliases["primary"])
}
@@ -563,43 +557,13 @@ func TestSwitchingScenarios(t *testing.T) {
})
}
func TestToolCapabilityFiltering(t *testing.T) {
t.Run("all models checked for tool capability", func(t *testing.T) {
// Both cloud and local models are checked for tool capability via Show API
// Only models with "tools" in capabilities are included
m := modelInfo{Name: "tool-model", Remote: false, ToolCapable: true}
if !m.ToolCapable {
t.Error("tool capable model should be marked as such")
}
})
t.Run("modelInfo includes ToolCapable field", func(t *testing.T) {
m := modelInfo{Name: "test", Remote: true, ToolCapable: true}
if !m.ToolCapable {
t.Error("ToolCapable field should be accessible")
}
})
}
func TestIsCloudModel_RequiresClient(t *testing.T) {
t.Run("nil client always returns false", func(t *testing.T) {
// isCloudModel now only uses Show API, no suffix detection
if isCloudModel(context.Background(), nil, "model:cloud") {
t.Error("nil client should return false regardless of suffix")
}
if isCloudModel(context.Background(), nil, "local-model") {
t.Error("nil client should return false")
}
})
}
func TestModelsAndAliasesMustStayInSync(t *testing.T) {
t.Run("saveAliases followed by saveIntegration keeps them in sync", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
// Save aliases with one model
if err := saveAliases("claude", map[string]string{"primary": "model-a"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model-a"}); err != nil {
t.Fatal(err)
}
@@ -608,7 +572,7 @@ func TestModelsAndAliasesMustStayInSync(t *testing.T) {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
if loaded.Aliases["primary"] != loaded.Models[0] {
t.Errorf("aliases.primary (%q) != models[0] (%q)", loaded.Aliases["primary"], loaded.Models[0])
}
@@ -622,11 +586,11 @@ func TestModelsAndAliasesMustStayInSync(t *testing.T) {
if err := SaveIntegration("claude", []string{"old-model"}); err != nil {
t.Fatal(err)
}
if err := saveAliases("claude", map[string]string{"primary": "new-model"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "new-model"}); err != nil {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
// They should be different (this is the bug state)
if loaded.Models[0] == loaded.Aliases["primary"] {
@@ -638,7 +602,7 @@ func TestModelsAndAliasesMustStayInSync(t *testing.T) {
t.Fatal(err)
}
loaded, _ = loadIntegration("claude")
loaded, _ = LoadIntegration("claude")
if loaded.Models[0] != loaded.Aliases["primary"] {
t.Errorf("after fix: models[0] (%q) should equal aliases.primary (%q)",
loaded.Models[0], loaded.Aliases["primary"])
@@ -653,20 +617,20 @@ func TestModelsAndAliasesMustStayInSync(t *testing.T) {
if err := SaveIntegration("claude", []string{"initial-model"}); err != nil {
t.Fatal(err)
}
if err := saveAliases("claude", map[string]string{"primary": "initial-model"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "initial-model"}); err != nil {
t.Fatal(err)
}
// Update aliases AND models together
newAliases := map[string]string{"primary": "updated-model"}
if err := saveAliases("claude", newAliases); err != nil {
if err := SaveAliases("claude", newAliases); err != nil {
t.Fatal(err)
}
if err := SaveIntegration("claude", []string{newAliases["primary"]}); err != nil {
t.Fatal(err)
}
loaded, _ := loadIntegration("claude")
loaded, _ := LoadIntegration("claude")
if loaded.Models[0] != "updated-model" {
t.Errorf("models[0] should be updated-model, got %q", loaded.Models[0])
}
+13 -78
View File
@@ -10,17 +10,10 @@ import (
// setTestHome sets both HOME (Unix) and USERPROFILE (Windows) for cross-platform tests
func setTestHome(t *testing.T, dir string) {
t.Setenv("HOME", dir)
t.Setenv("TMPDIR", dir)
t.Setenv("USERPROFILE", dir)
}
// editorPaths is a test helper that safely calls Paths if the runner implements Editor
func editorPaths(r Runner) []string {
if editor, ok := r.(Editor); ok {
return editor.Paths()
}
return nil
}
func TestIntegrationConfig(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
@@ -31,7 +24,7 @@ func TestIntegrationConfig(t *testing.T) {
t.Fatal(err)
}
config, err := loadIntegration("claude")
config, err := LoadIntegration("claude")
if err != nil {
t.Fatal(err)
}
@@ -55,11 +48,11 @@ func TestIntegrationConfig(t *testing.T) {
"primary": "llama3.2:70b",
"fast": "llama3.2:8b",
}
if err := saveAliases("claude", aliases); err != nil {
if err := SaveAliases("claude", aliases); err != nil {
t.Fatal(err)
}
config, err := loadIntegration("claude")
config, err := LoadIntegration("claude")
if err != nil {
t.Fatal(err)
}
@@ -77,14 +70,14 @@ func TestIntegrationConfig(t *testing.T) {
if err := SaveIntegration("claude", []string{"model-a"}); err != nil {
t.Fatal(err)
}
if err := saveAliases("claude", map[string]string{"primary": "model-a", "fast": "model-small"}); err != nil {
if err := SaveAliases("claude", map[string]string{"primary": "model-a", "fast": "model-small"}); err != nil {
t.Fatal(err)
}
if err := SaveIntegration("claude", []string{"model-b"}); err != nil {
t.Fatal(err)
}
config, err := loadIntegration("claude")
config, err := LoadIntegration("claude")
if err != nil {
t.Fatal(err)
}
@@ -96,7 +89,7 @@ func TestIntegrationConfig(t *testing.T) {
t.Run("defaultModel returns first model", func(t *testing.T) {
SaveIntegration("codex", []string{"model-a", "model-b"})
config, _ := loadIntegration("codex")
config, _ := LoadIntegration("codex")
defaultModel := ""
if len(config.Models) > 0 {
defaultModel = config.Models[0]
@@ -120,7 +113,7 @@ func TestIntegrationConfig(t *testing.T) {
t.Run("app name is case-insensitive", func(t *testing.T) {
SaveIntegration("Claude", []string{"model-x"})
config, err := loadIntegration("claude")
config, err := LoadIntegration("claude")
if err != nil {
t.Fatal(err)
}
@@ -137,8 +130,8 @@ func TestIntegrationConfig(t *testing.T) {
SaveIntegration("app1", []string{"model-1"})
SaveIntegration("app2", []string{"model-2"})
config1, _ := loadIntegration("app1")
config2, _ := loadIntegration("app2")
config1, _ := LoadIntegration("app1")
config2, _ := LoadIntegration("app2")
defaultModel1 := ""
if len(config1.Models) > 0 {
@@ -185,64 +178,6 @@ func TestListIntegrations(t *testing.T) {
})
}
func TestEditorPaths(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Run("returns empty for claude (no Editor)", func(t *testing.T) {
r := integrations["claude"]
paths := editorPaths(r)
if len(paths) != 0 {
t.Errorf("expected no paths for claude, got %v", paths)
}
})
t.Run("returns empty for codex (no Editor)", func(t *testing.T) {
r := integrations["codex"]
paths := editorPaths(r)
if len(paths) != 0 {
t.Errorf("expected no paths for codex, got %v", paths)
}
})
t.Run("returns empty for droid when no config exists", func(t *testing.T) {
r := integrations["droid"]
paths := editorPaths(r)
if len(paths) != 0 {
t.Errorf("expected no paths, got %v", paths)
}
})
t.Run("returns path for droid when config exists", func(t *testing.T) {
settingsDir, _ := os.UserHomeDir()
settingsDir = filepath.Join(settingsDir, ".factory")
os.MkdirAll(settingsDir, 0o755)
os.WriteFile(filepath.Join(settingsDir, "settings.json"), []byte(`{}`), 0o644)
r := integrations["droid"]
paths := editorPaths(r)
if len(paths) != 1 {
t.Errorf("expected 1 path, got %d", len(paths))
}
})
t.Run("returns paths for opencode when configs exist", func(t *testing.T) {
home, _ := os.UserHomeDir()
configDir := filepath.Join(home, ".config", "opencode")
stateDir := filepath.Join(home, ".local", "state", "opencode")
os.MkdirAll(configDir, 0o755)
os.MkdirAll(stateDir, 0o755)
os.WriteFile(filepath.Join(configDir, "opencode.json"), []byte(`{}`), 0o644)
os.WriteFile(filepath.Join(stateDir, "model.json"), []byte(`{}`), 0o644)
r := integrations["opencode"]
paths := editorPaths(r)
if len(paths) != 2 {
t.Errorf("expected 2 paths, got %d: %v", len(paths), paths)
}
})
}
func TestLoadIntegration_CorruptedJSON(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
@@ -251,7 +186,7 @@ func TestLoadIntegration_CorruptedJSON(t *testing.T) {
os.MkdirAll(dir, 0o755)
os.WriteFile(filepath.Join(dir, "config.json"), []byte(`{corrupted json`), 0o644)
_, err := loadIntegration("test")
_, err := LoadIntegration("test")
if err == nil {
t.Error("expected error for nonexistent integration in corrupted file")
}
@@ -265,7 +200,7 @@ func TestSaveIntegration_NilModels(t *testing.T) {
t.Fatalf("saveIntegration with nil models failed: %v", err)
}
config, err := loadIntegration("test")
config, err := LoadIntegration("test")
if err != nil {
t.Fatalf("loadIntegration failed: %v", err)
}
@@ -294,7 +229,7 @@ func TestLoadIntegration_NonexistentIntegration(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
_, err := loadIntegration("nonexistent")
_, err := LoadIntegration("nonexistent")
if err == nil {
t.Error("expected error for nonexistent integration, got nil")
}
-99
View File
@@ -1,99 +0,0 @@
package config
import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"time"
)
func readJSONFile(path string) (map[string]any, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var result map[string]any
if err := json.Unmarshal(data, &result); err != nil {
return nil, err
}
return result, nil
}
func copyFile(src, dst string) error {
info, err := os.Stat(src)
if err != nil {
return err
}
data, err := os.ReadFile(src)
if err != nil {
return err
}
return os.WriteFile(dst, data, info.Mode().Perm())
}
func backupDir() string {
return filepath.Join(os.TempDir(), "ollama-backups")
}
func backupToTmp(srcPath string) (string, error) {
dir := backupDir()
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
backupPath := filepath.Join(dir, fmt.Sprintf("%s.%d", filepath.Base(srcPath), time.Now().Unix()))
if err := copyFile(srcPath, backupPath); err != nil {
return "", err
}
return backupPath, nil
}
// writeWithBackup writes data to path via temp file + rename, backing up any existing file first
func writeWithBackup(path string, data []byte) error {
var backupPath string
// backup must be created before any writes to the target file
if existingContent, err := os.ReadFile(path); err == nil {
if !bytes.Equal(existingContent, data) {
backupPath, err = backupToTmp(path)
if err != nil {
return fmt.Errorf("backup failed: %w", err)
}
}
} else if !os.IsNotExist(err) {
return fmt.Errorf("read existing file: %w", err)
}
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, ".tmp-*")
if err != nil {
return fmt.Errorf("create temp failed: %w", err)
}
tmpPath := tmp.Name()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("write failed: %w", err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("sync failed: %w", err)
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("close failed: %w", err)
}
if err := os.Rename(tmpPath, path); err != nil {
_ = os.Remove(tmpPath)
if backupPath != "" {
_ = copyFile(backupPath, path)
}
return fmt.Errorf("rename failed: %w", err)
}
return nil
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-279
View File
@@ -1,279 +0,0 @@
package config
import (
"context"
"encoding/json"
"errors"
"fmt"
"maps"
"os"
"os/exec"
"path/filepath"
"slices"
"strings"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
)
// OpenCode implements Runner and Editor for OpenCode integration
type OpenCode struct{}
// cloudModelLimit holds context and output token limits for a cloud model.
type cloudModelLimit struct {
Context int
Output int
}
// lookupCloudModelLimit returns the token limits for a cloud model.
// It tries the exact name first, then strips the ":cloud" suffix.
func lookupCloudModelLimit(name string) (cloudModelLimit, bool) {
if l, ok := cloudModelLimits[name]; ok {
return l, true
}
base := strings.TrimSuffix(name, ":cloud")
if base != name {
if l, ok := cloudModelLimits[base]; ok {
return l, true
}
}
return cloudModelLimit{}, false
}
func (o *OpenCode) String() string { return "OpenCode" }
func (o *OpenCode) Run(model string, args []string) error {
if _, err := exec.LookPath("opencode"); err != nil {
return fmt.Errorf("opencode is not installed, install from https://opencode.ai")
}
// Call Edit() to ensure config is up-to-date before launch
models := []string{model}
if config, err := loadIntegration("opencode"); err == nil && len(config.Models) > 0 {
models = config.Models
}
var err error
models, err = resolveEditorModels("opencode", models, func() ([]string, error) {
return selectModels(context.Background(), "opencode", "")
})
if errors.Is(err, errCancelled) {
return nil
}
if err != nil {
return err
}
if err := o.Edit(models); err != nil {
return fmt.Errorf("setup failed: %w", err)
}
cmd := exec.Command("opencode", args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func (o *OpenCode) Paths() []string {
home, err := os.UserHomeDir()
if err != nil {
return nil
}
var paths []string
p := filepath.Join(home, ".config", "opencode", "opencode.json")
if _, err := os.Stat(p); err == nil {
paths = append(paths, p)
}
sp := filepath.Join(home, ".local", "state", "opencode", "model.json")
if _, err := os.Stat(sp); err == nil {
paths = append(paths, sp)
}
return paths
}
func (o *OpenCode) Edit(modelList []string) error {
if len(modelList) == 0 {
return nil
}
home, err := os.UserHomeDir()
if err != nil {
return err
}
configPath := filepath.Join(home, ".config", "opencode", "opencode.json")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
return err
}
config := make(map[string]any)
if data, err := os.ReadFile(configPath); err == nil {
_ = json.Unmarshal(data, &config) // Ignore parse errors; treat missing/corrupt files as empty
}
config["$schema"] = "https://opencode.ai/config.json"
provider, ok := config["provider"].(map[string]any)
if !ok {
provider = make(map[string]any)
}
ollama, ok := provider["ollama"].(map[string]any)
if !ok {
ollama = map[string]any{
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": map[string]any{
"baseURL": envconfig.Host().String() + "/v1",
},
}
}
models, ok := ollama["models"].(map[string]any)
if !ok {
models = make(map[string]any)
}
selectedSet := make(map[string]bool)
for _, m := range modelList {
selectedSet[m] = true
}
for name, cfg := range models {
if cfgMap, ok := cfg.(map[string]any); ok {
if isOllamaModel(cfgMap) && !selectedSet[name] {
delete(models, name)
}
}
}
client, _ := api.ClientFromEnvironment()
for _, model := range modelList {
if existing, ok := models[model].(map[string]any); ok {
// migrate existing models without _launch marker
if isOllamaModel(existing) {
existing["_launch"] = true
if name, ok := existing["name"].(string); ok {
existing["name"] = strings.TrimSuffix(name, " [Ollama]")
}
}
if isCloudModel(context.Background(), client, model) {
if l, ok := lookupCloudModelLimit(model); ok {
existing["limit"] = map[string]any{
"context": l.Context,
"output": l.Output,
}
}
}
continue
}
entry := map[string]any{
"name": model,
"_launch": true,
}
if isCloudModel(context.Background(), client, model) {
if l, ok := lookupCloudModelLimit(model); ok {
entry["limit"] = map[string]any{
"context": l.Context,
"output": l.Output,
}
}
}
models[model] = entry
}
ollama["models"] = models
provider["ollama"] = ollama
config["provider"] = provider
configData, err := json.MarshalIndent(config, "", " ")
if err != nil {
return err
}
if err := writeWithBackup(configPath, configData); err != nil {
return err
}
statePath := filepath.Join(home, ".local", "state", "opencode", "model.json")
if err := os.MkdirAll(filepath.Dir(statePath), 0o755); err != nil {
return err
}
state := map[string]any{
"recent": []any{},
"favorite": []any{},
"variant": map[string]any{},
}
if data, err := os.ReadFile(statePath); err == nil {
_ = json.Unmarshal(data, &state) // Ignore parse errors; use defaults
}
recent, _ := state["recent"].([]any)
modelSet := make(map[string]bool)
for _, m := range modelList {
modelSet[m] = true
}
// Filter out existing Ollama models we're about to re-add
newRecent := slices.DeleteFunc(slices.Clone(recent), func(entry any) bool {
e, ok := entry.(map[string]any)
if !ok || e["providerID"] != "ollama" {
return false
}
modelID, _ := e["modelID"].(string)
return modelSet[modelID]
})
// Prepend models in reverse order so first model ends up first
for _, model := range slices.Backward(modelList) {
newRecent = slices.Insert(newRecent, 0, any(map[string]any{
"providerID": "ollama",
"modelID": model,
}))
}
const maxRecentModels = 10
newRecent = newRecent[:min(len(newRecent), maxRecentModels)]
state["recent"] = newRecent
stateData, err := json.MarshalIndent(state, "", " ")
if err != nil {
return err
}
return writeWithBackup(statePath, stateData)
}
func (o *OpenCode) Models() []string {
home, err := os.UserHomeDir()
if err != nil {
return nil
}
config, err := readJSONFile(filepath.Join(home, ".config", "opencode", "opencode.json"))
if err != nil {
return nil
}
provider, _ := config["provider"].(map[string]any)
ollama, _ := provider["ollama"].(map[string]any)
models, _ := ollama["models"].(map[string]any)
if len(models) == 0 {
return nil
}
keys := slices.Collect(maps.Keys(models))
slices.Sort(keys)
return keys
}
// isOllamaModel reports whether a model config entry is managed by us
func isOllamaModel(cfg map[string]any) bool {
if v, ok := cfg["_launch"].(bool); ok && v {
return true
}
// previously used [Ollama] as a suffix for the model managed by ollama launch
if name, ok := cfg["name"].(string); ok {
return strings.HasSuffix(name, "[Ollama]")
}
return false
}
-668
View File
@@ -1,668 +0,0 @@
package config
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"testing"
)
func TestOpenCodeIntegration(t *testing.T) {
o := &OpenCode{}
t.Run("String", func(t *testing.T) {
if got := o.String(); got != "OpenCode" {
t.Errorf("String() = %q, want %q", got, "OpenCode")
}
})
t.Run("implements Runner", func(t *testing.T) {
var _ Runner = o
})
t.Run("implements Editor", func(t *testing.T) {
var _ Editor = o
})
}
func TestOpenCodeEdit(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
stateDir := filepath.Join(tmpDir, ".local", "state", "opencode")
statePath := filepath.Join(stateDir, "model.json")
cleanup := func() {
os.RemoveAll(configDir)
os.RemoveAll(stateDir)
}
t.Run("fresh install", func(t *testing.T) {
cleanup()
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
assertOpenCodeModelExists(t, configPath, "llama3.2")
assertOpenCodeRecentModel(t, statePath, 0, "ollama", "llama3.2")
})
t.Run("preserve other providers", func(t *testing.T) {
cleanup()
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{"provider":{"anthropic":{"apiKey":"xxx"}}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
var cfg map[string]any
json.Unmarshal(data, &cfg)
provider := cfg["provider"].(map[string]any)
if provider["anthropic"] == nil {
t.Error("anthropic provider was removed")
}
assertOpenCodeModelExists(t, configPath, "llama3.2")
})
t.Run("preserve other models", func(t *testing.T) {
cleanup()
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{"provider":{"ollama":{"models":{"mistral":{"name":"Mistral"}}}}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
assertOpenCodeModelExists(t, configPath, "mistral")
assertOpenCodeModelExists(t, configPath, "llama3.2")
})
t.Run("update existing model", func(t *testing.T) {
cleanup()
o.Edit([]string{"llama3.2"})
o.Edit([]string{"llama3.2"})
assertOpenCodeModelExists(t, configPath, "llama3.2")
})
t.Run("preserve top-level keys", func(t *testing.T) {
cleanup()
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{"theme":"dark","keybindings":{}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
var cfg map[string]any
json.Unmarshal(data, &cfg)
if cfg["theme"] != "dark" {
t.Error("theme was removed")
}
if cfg["keybindings"] == nil {
t.Error("keybindings was removed")
}
})
t.Run("model state - insert at index 0", func(t *testing.T) {
cleanup()
os.MkdirAll(stateDir, 0o755)
os.WriteFile(statePath, []byte(`{"recent":[{"providerID":"anthropic","modelID":"claude"}],"favorite":[],"variant":{}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
assertOpenCodeRecentModel(t, statePath, 0, "ollama", "llama3.2")
assertOpenCodeRecentModel(t, statePath, 1, "anthropic", "claude")
})
t.Run("model state - preserve favorites and variants", func(t *testing.T) {
cleanup()
os.MkdirAll(stateDir, 0o755)
os.WriteFile(statePath, []byte(`{"recent":[],"favorite":[{"providerID":"x","modelID":"y"}],"variant":{"a":"b"}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(statePath)
var state map[string]any
json.Unmarshal(data, &state)
if len(state["favorite"].([]any)) != 1 {
t.Error("favorite was modified")
}
if state["variant"].(map[string]any)["a"] != "b" {
t.Error("variant was modified")
}
})
t.Run("model state - deduplicate on re-add", func(t *testing.T) {
cleanup()
os.MkdirAll(stateDir, 0o755)
os.WriteFile(statePath, []byte(`{"recent":[{"providerID":"ollama","modelID":"llama3.2"},{"providerID":"anthropic","modelID":"claude"}],"favorite":[],"variant":{}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(statePath)
var state map[string]any
json.Unmarshal(data, &state)
recent := state["recent"].([]any)
if len(recent) != 2 {
t.Errorf("expected 2 recent entries, got %d", len(recent))
}
assertOpenCodeRecentModel(t, statePath, 0, "ollama", "llama3.2")
})
t.Run("remove model", func(t *testing.T) {
cleanup()
// First add two models
o.Edit([]string{"llama3.2", "mistral"})
assertOpenCodeModelExists(t, configPath, "llama3.2")
assertOpenCodeModelExists(t, configPath, "mistral")
// Then remove one by only selecting the other
o.Edit([]string{"llama3.2"})
assertOpenCodeModelExists(t, configPath, "llama3.2")
assertOpenCodeModelNotExists(t, configPath, "mistral")
})
t.Run("preserve user customizations on managed models", func(t *testing.T) {
cleanup()
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
// Add custom fields to the model entry (simulating user edits)
data, _ := os.ReadFile(configPath)
var cfg map[string]any
json.Unmarshal(data, &cfg)
provider := cfg["provider"].(map[string]any)
ollama := provider["ollama"].(map[string]any)
models := ollama["models"].(map[string]any)
entry := models["llama3.2"].(map[string]any)
entry["_myPref"] = "custom-value"
entry["_myNum"] = 42
configData, _ := json.MarshalIndent(cfg, "", " ")
os.WriteFile(configPath, configData, 0o644)
// Re-run Edit — should preserve custom fields
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ = os.ReadFile(configPath)
json.Unmarshal(data, &cfg)
provider = cfg["provider"].(map[string]any)
ollama = provider["ollama"].(map[string]any)
models = ollama["models"].(map[string]any)
entry = models["llama3.2"].(map[string]any)
if entry["_myPref"] != "custom-value" {
t.Errorf("_myPref was lost: got %v", entry["_myPref"])
}
if entry["_myNum"] != float64(42) {
t.Errorf("_myNum was lost: got %v", entry["_myNum"])
}
if v, ok := entry["_launch"].(bool); !ok || !v {
t.Errorf("_launch marker missing or false: got %v", entry["_launch"])
}
})
t.Run("migrate legacy [Ollama] suffix entries", func(t *testing.T) {
cleanup()
// Write a config with a legacy entry (has [Ollama] suffix but no _launch marker)
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{"provider":{"ollama":{"models":{"llama3.2":{"name":"llama3.2 [Ollama]"}}}}}`), 0o644)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
var cfg map[string]any
json.Unmarshal(data, &cfg)
provider := cfg["provider"].(map[string]any)
ollama := provider["ollama"].(map[string]any)
models := ollama["models"].(map[string]any)
entry := models["llama3.2"].(map[string]any)
// _launch marker should be added
if v, ok := entry["_launch"].(bool); !ok || !v {
t.Errorf("_launch marker not added during migration: got %v", entry["_launch"])
}
// [Ollama] suffix should be stripped
if name, ok := entry["name"].(string); !ok || name != "llama3.2" {
t.Errorf("name suffix not stripped: got %q", entry["name"])
}
})
t.Run("remove model preserves non-ollama models", func(t *testing.T) {
cleanup()
os.MkdirAll(configDir, 0o755)
// Add a non-Ollama model manually
os.WriteFile(configPath, []byte(`{"provider":{"ollama":{"models":{"external":{"name":"External Model"}}}}}`), 0o644)
o.Edit([]string{"llama3.2"})
assertOpenCodeModelExists(t, configPath, "llama3.2")
assertOpenCodeModelExists(t, configPath, "external") // Should be preserved
})
}
func assertOpenCodeModelExists(t *testing.T, path, model string) {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
t.Fatal(err)
}
provider, ok := cfg["provider"].(map[string]any)
if !ok {
t.Fatal("provider not found")
}
ollama, ok := provider["ollama"].(map[string]any)
if !ok {
t.Fatal("ollama provider not found")
}
models, ok := ollama["models"].(map[string]any)
if !ok {
t.Fatal("models not found")
}
if models[model] == nil {
t.Errorf("model %s not found", model)
}
}
func assertOpenCodeModelNotExists(t *testing.T, path, model string) {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
t.Fatal(err)
}
provider, ok := cfg["provider"].(map[string]any)
if !ok {
return // No provider means no model
}
ollama, ok := provider["ollama"].(map[string]any)
if !ok {
return // No ollama means no model
}
models, ok := ollama["models"].(map[string]any)
if !ok {
return // No models means no model
}
if models[model] != nil {
t.Errorf("model %s should not exist but was found", model)
}
}
func assertOpenCodeRecentModel(t *testing.T, path string, index int, providerID, modelID string) {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var state map[string]any
if err := json.Unmarshal(data, &state); err != nil {
t.Fatal(err)
}
recent, ok := state["recent"].([]any)
if !ok {
t.Fatal("recent not found")
}
if index >= len(recent) {
t.Fatalf("index %d out of range (len=%d)", index, len(recent))
}
entry, ok := recent[index].(map[string]any)
if !ok {
t.Fatal("entry is not a map")
}
if entry["providerID"] != providerID {
t.Errorf("expected providerID %s, got %s", providerID, entry["providerID"])
}
if entry["modelID"] != modelID {
t.Errorf("expected modelID %s, got %s", modelID, entry["modelID"])
}
}
// Edge case tests for opencode.go
func TestOpenCodeEdit_CorruptedConfigJSON(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{corrupted json content`), 0o644)
// Should not panic - corrupted JSON should be treated as empty
err := o.Edit([]string{"llama3.2"})
if err != nil {
t.Fatalf("Edit failed with corrupted config: %v", err)
}
// Verify valid JSON was created
data, _ := os.ReadFile(configPath)
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
t.Errorf("resulting config is not valid JSON: %v", err)
}
}
func TestOpenCodeEdit_CorruptedStateJSON(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
stateDir := filepath.Join(tmpDir, ".local", "state", "opencode")
statePath := filepath.Join(stateDir, "model.json")
os.MkdirAll(stateDir, 0o755)
os.WriteFile(statePath, []byte(`{corrupted state`), 0o644)
err := o.Edit([]string{"llama3.2"})
if err != nil {
t.Fatalf("Edit failed with corrupted state: %v", err)
}
// Verify valid state was created
data, _ := os.ReadFile(statePath)
var state map[string]any
if err := json.Unmarshal(data, &state); err != nil {
t.Errorf("resulting state is not valid JSON: %v", err)
}
}
func TestOpenCodeEdit_WrongTypeProvider(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{"provider": "not a map"}`), 0o644)
err := o.Edit([]string{"llama3.2"})
if err != nil {
t.Fatalf("Edit with wrong type provider failed: %v", err)
}
// Verify provider is now correct type
data, _ := os.ReadFile(configPath)
var cfg map[string]any
json.Unmarshal(data, &cfg)
provider, ok := cfg["provider"].(map[string]any)
if !ok {
t.Fatalf("provider should be map after setup, got %T", cfg["provider"])
}
if provider["ollama"] == nil {
t.Error("ollama provider should be created")
}
}
func TestOpenCodeEdit_WrongTypeRecent(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
stateDir := filepath.Join(tmpDir, ".local", "state", "opencode")
statePath := filepath.Join(stateDir, "model.json")
os.MkdirAll(stateDir, 0o755)
os.WriteFile(statePath, []byte(`{"recent": "not an array", "favorite": [], "variant": {}}`), 0o644)
err := o.Edit([]string{"llama3.2"})
if err != nil {
t.Fatalf("Edit with wrong type recent failed: %v", err)
}
// The function should handle this gracefully
data, _ := os.ReadFile(statePath)
var state map[string]any
json.Unmarshal(data, &state)
// recent should be properly set after setup
recent, ok := state["recent"].([]any)
if !ok {
t.Logf("Note: recent type after setup is %T (documenting behavior)", state["recent"])
} else if len(recent) == 0 {
t.Logf("Note: recent is empty (documenting behavior)")
}
}
func TestOpenCodeEdit_EmptyModels(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
os.MkdirAll(configDir, 0o755)
originalContent := `{"provider":{"ollama":{"models":{"existing":{}}}}}`
os.WriteFile(configPath, []byte(originalContent), 0o644)
// Empty models should be no-op
err := o.Edit([]string{})
if err != nil {
t.Fatalf("Edit with empty models failed: %v", err)
}
// Original content should be preserved (file not modified)
data, _ := os.ReadFile(configPath)
if string(data) != originalContent {
t.Errorf("empty models should not modify file, but content changed")
}
}
func TestOpenCodeEdit_SpecialCharsInModelName(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
// Model name with special characters (though unusual)
specialModel := `model-with-"quotes"`
err := o.Edit([]string{specialModel})
if err != nil {
t.Fatalf("Edit with special chars failed: %v", err)
}
// Verify it was stored correctly
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
data, _ := os.ReadFile(configPath)
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
t.Fatalf("resulting config is invalid JSON: %v", err)
}
// Model should be accessible
provider, _ := cfg["provider"].(map[string]any)
ollama, _ := provider["ollama"].(map[string]any)
models, _ := ollama["models"].(map[string]any)
if models[specialModel] == nil {
t.Errorf("model with special chars not found in config")
}
}
func readOpenCodeModel(t *testing.T, configPath, model string) map[string]any {
t.Helper()
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
var cfg map[string]any
json.Unmarshal(data, &cfg)
provider := cfg["provider"].(map[string]any)
ollama := provider["ollama"].(map[string]any)
models := ollama["models"].(map[string]any)
entry, ok := models[model].(map[string]any)
if !ok {
t.Fatalf("model %s not found in config", model)
}
return entry
}
func TestOpenCodeEdit_LocalModelNoLimit(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configPath := filepath.Join(tmpDir, ".config", "opencode", "opencode.json")
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
entry := readOpenCodeModel(t, configPath, "llama3.2")
if entry["limit"] != nil {
t.Errorf("local model should not have limit set, got %v", entry["limit"])
}
}
func TestOpenCodeEdit_PreservesUserLimit(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
// Set up a model with a user-configured limit
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(`{
"provider": {
"ollama": {
"models": {
"llama3.2": {
"name": "llama3.2",
"_launch": true,
"limit": {"context": 8192, "output": 4096}
}
}
}
}
}`), 0o644)
// Re-edit should preserve the user's limit (not delete it)
if err := o.Edit([]string{"llama3.2"}); err != nil {
t.Fatal(err)
}
entry := readOpenCodeModel(t, configPath, "llama3.2")
limit, ok := entry["limit"].(map[string]any)
if !ok {
t.Fatal("user-configured limit was removed")
}
if limit["context"] != float64(8192) {
t.Errorf("context limit changed: got %v, want 8192", limit["context"])
}
if limit["output"] != float64(4096) {
t.Errorf("output limit changed: got %v, want 4096", limit["output"])
}
}
func TestOpenCodeEdit_CloudModelLimitStructure(t *testing.T) {
// Verify that when a cloud model entry has limits set (as Edit would do),
// the structure matches what opencode expects and re-edit preserves them.
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
configDir := filepath.Join(tmpDir, ".config", "opencode")
configPath := filepath.Join(configDir, "opencode.json")
expected := cloudModelLimits["glm-4.7"]
// Simulate a cloud model that already has the limit set by a previous Edit
os.MkdirAll(configDir, 0o755)
os.WriteFile(configPath, []byte(fmt.Sprintf(`{
"provider": {
"ollama": {
"models": {
"glm-4.7:cloud": {
"name": "glm-4.7:cloud",
"_launch": true,
"limit": {"context": %d, "output": %d}
}
}
}
}
}`, expected.Context, expected.Output)), 0o644)
// Re-edit should preserve the cloud model limit
if err := o.Edit([]string{"glm-4.7:cloud"}); err != nil {
t.Fatal(err)
}
entry := readOpenCodeModel(t, configPath, "glm-4.7:cloud")
limit, ok := entry["limit"].(map[string]any)
if !ok {
t.Fatal("cloud model limit was removed on re-edit")
}
if limit["context"] != float64(expected.Context) {
t.Errorf("context = %v, want %d", limit["context"], expected.Context)
}
if limit["output"] != float64(expected.Output) {
t.Errorf("output = %v, want %d", limit["output"], expected.Output)
}
}
func TestLookupCloudModelLimit(t *testing.T) {
tests := []struct {
name string
wantOK bool
wantContext int
wantOutput int
}{
{"glm-4.7", true, 202_752, 131_072},
{"glm-4.7:cloud", true, 202_752, 131_072},
{"kimi-k2.5", true, 262_144, 262_144},
{"kimi-k2.5:cloud", true, 262_144, 262_144},
{"deepseek-v3.2", true, 163_840, 65_536},
{"deepseek-v3.2:cloud", true, 163_840, 65_536},
{"qwen3-coder:480b", true, 262_144, 65_536},
{"qwen3-coder-next:cloud", true, 262_144, 32_768},
{"llama3.2", false, 0, 0},
{"unknown-model:cloud", false, 0, 0},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
l, ok := lookupCloudModelLimit(tt.name)
if ok != tt.wantOK {
t.Errorf("lookupCloudModelLimit(%q) ok = %v, want %v", tt.name, ok, tt.wantOK)
}
if ok {
if l.Context != tt.wantContext {
t.Errorf("context = %d, want %d", l.Context, tt.wantContext)
}
if l.Output != tt.wantOutput {
t.Errorf("output = %d, want %d", l.Output, tt.wantOutput)
}
}
})
}
}
func TestOpenCodeModels_NoConfig(t *testing.T) {
o := &OpenCode{}
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
models := o.Models()
if len(models) > 0 {
t.Errorf("expected nil/empty for missing config, got %v", models)
}
}
-237
View File
@@ -1,237 +0,0 @@
package config
import (
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"os/exec"
"path/filepath"
"slices"
"strings"
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/types/model"
)
// Pi implements Runner and Editor for Pi (Pi Coding Agent) integration
type Pi struct{}
func (p *Pi) String() string { return "Pi" }
func (p *Pi) Run(model string, args []string) error {
if _, err := exec.LookPath("pi"); err != nil {
return fmt.Errorf("pi is not installed, install with: npm install -g @mariozechner/pi-coding-agent")
}
// Call Edit() to ensure config is up-to-date before launch
models := []string{model}
if config, err := loadIntegration("pi"); err == nil && len(config.Models) > 0 {
models = config.Models
}
if err := p.Edit(models); err != nil {
return fmt.Errorf("setup failed: %w", err)
}
cmd := exec.Command("pi", args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func (p *Pi) Paths() []string {
home, err := os.UserHomeDir()
if err != nil {
return nil
}
var paths []string
modelsPath := filepath.Join(home, ".pi", "agent", "models.json")
if _, err := os.Stat(modelsPath); err == nil {
paths = append(paths, modelsPath)
}
settingsPath := filepath.Join(home, ".pi", "agent", "settings.json")
if _, err := os.Stat(settingsPath); err == nil {
paths = append(paths, settingsPath)
}
return paths
}
func (p *Pi) Edit(models []string) error {
if len(models) == 0 {
return nil
}
home, err := os.UserHomeDir()
if err != nil {
return err
}
configPath := filepath.Join(home, ".pi", "agent", "models.json")
if err := os.MkdirAll(filepath.Dir(configPath), 0o755); err != nil {
return err
}
config := make(map[string]any)
if data, err := os.ReadFile(configPath); err == nil {
_ = json.Unmarshal(data, &config)
}
providers, ok := config["providers"].(map[string]any)
if !ok {
providers = make(map[string]any)
}
ollama, ok := providers["ollama"].(map[string]any)
if !ok {
ollama = map[string]any{
"baseUrl": envconfig.Host().String() + "/v1",
"api": "openai-completions",
"apiKey": "ollama",
}
}
existingModels, ok := ollama["models"].([]any)
if !ok {
existingModels = make([]any, 0)
}
// Build set of selected models to track which need to be added
selectedSet := make(map[string]bool, len(models))
for _, m := range models {
selectedSet[m] = true
}
// Build new models list:
// 1. Keep user-managed models (no _launch marker) - untouched
// 2. Keep ollama-managed models (_launch marker) that are still selected
// 3. Add new ollama-managed models
var newModels []any
for _, m := range existingModels {
if modelObj, ok := m.(map[string]any); ok {
if id, ok := modelObj["id"].(string); ok {
// User-managed model (no _launch marker) - always preserve
if !isPiOllamaModel(modelObj) {
newModels = append(newModels, m)
} else if selectedSet[id] {
// Ollama-managed and still selected - keep it
newModels = append(newModels, m)
selectedSet[id] = false
}
}
}
}
// Add newly selected models that weren't already in the list
client := api.NewClient(envconfig.Host(), http.DefaultClient)
ctx := context.Background()
for _, model := range models {
if selectedSet[model] {
newModels = append(newModels, createConfig(ctx, client, model))
}
}
ollama["models"] = newModels
providers["ollama"] = ollama
config["providers"] = providers
configData, err := json.MarshalIndent(config, "", " ")
if err != nil {
return err
}
if err := writeWithBackup(configPath, configData); err != nil {
return err
}
// Update settings.json with default provider and model
settingsPath := filepath.Join(home, ".pi", "agent", "settings.json")
settings := make(map[string]any)
if data, err := os.ReadFile(settingsPath); err == nil {
_ = json.Unmarshal(data, &settings)
}
settings["defaultProvider"] = "ollama"
settings["defaultModel"] = models[0]
settingsData, err := json.MarshalIndent(settings, "", " ")
if err != nil {
return err
}
return writeWithBackup(settingsPath, settingsData)
}
func (p *Pi) Models() []string {
home, err := os.UserHomeDir()
if err != nil {
return nil
}
configPath := filepath.Join(home, ".pi", "agent", "models.json")
config, err := readJSONFile(configPath)
if err != nil {
return nil
}
providers, _ := config["providers"].(map[string]any)
ollama, _ := providers["ollama"].(map[string]any)
models, _ := ollama["models"].([]any)
var result []string
for _, m := range models {
if modelObj, ok := m.(map[string]any); ok {
if id, ok := modelObj["id"].(string); ok {
result = append(result, id)
}
}
}
slices.Sort(result)
return result
}
// isPiOllamaModel reports whether a model config entry is managed by ollama launch
func isPiOllamaModel(cfg map[string]any) bool {
if v, ok := cfg["_launch"].(bool); ok && v {
return true
}
return false
}
// createConfig builds Pi model config with capability detection
func createConfig(ctx context.Context, client *api.Client, modelID string) map[string]any {
cfg := map[string]any{
"id": modelID,
"_launch": true,
}
resp, err := client.Show(ctx, &api.ShowRequest{Model: modelID})
if err != nil {
return cfg
}
// Set input types based on vision capability
if slices.Contains(resp.Capabilities, model.CapabilityVision) {
cfg["input"] = []string{"text", "image"}
} else {
cfg["input"] = []string{"text"}
}
// Set reasoning based on thinking capability
if slices.Contains(resp.Capabilities, model.CapabilityThinking) {
cfg["reasoning"] = true
}
// Extract context window from ModelInfo
for key, val := range resp.ModelInfo {
if strings.HasSuffix(key, ".context_length") {
if ctxLen, ok := val.(float64); ok && ctxLen > 0 {
cfg["contextWindow"] = int(ctxLen)
}
break
}
}
return cfg
}
-59
View File
@@ -1,59 +0,0 @@
package config
import (
"errors"
"fmt"
"os"
"golang.org/x/term"
)
// ANSI escape sequences for terminal formatting.
const (
ansiBold = "\033[1m"
ansiReset = "\033[0m"
ansiGray = "\033[37m"
ansiGreen = "\033[32m"
ansiYellow = "\033[33m"
)
// ErrCancelled is returned when the user cancels a selection.
var ErrCancelled = errors.New("cancelled")
// errCancelled is kept as an alias for backward compatibility within the package.
var errCancelled = ErrCancelled
// DefaultConfirmPrompt provides a TUI-based confirmation prompt.
// When set, confirmPrompt delegates to it instead of using raw terminal I/O.
var DefaultConfirmPrompt func(prompt string) (bool, error)
func confirmPrompt(prompt string) (bool, error) {
if DefaultConfirmPrompt != nil {
return DefaultConfirmPrompt(prompt)
}
fd := int(os.Stdin.Fd())
oldState, err := term.MakeRaw(fd)
if err != nil {
return false, err
}
defer term.Restore(fd, oldState)
fmt.Fprintf(os.Stderr, "%s (\033[1my\033[0m/n) ", prompt)
buf := make([]byte, 1)
for {
if _, err := os.Stdin.Read(buf); err != nil {
return false, err
}
switch buf[0] {
case 'Y', 'y', 13:
fmt.Fprintf(os.Stderr, "yes\r\n")
return true, nil
case 'N', 'n', 27, 3:
fmt.Fprintf(os.Stderr, "no\r\n")
return false, nil
}
}
}
-19
View File
@@ -1,19 +0,0 @@
package config
import (
"testing"
)
func TestErrCancelled(t *testing.T) {
t.Run("NotNil", func(t *testing.T) {
if errCancelled == nil {
t.Error("errCancelled should not be nil")
}
})
t.Run("Message", func(t *testing.T) {
if errCancelled.Error() != "cancelled" {
t.Errorf("expected 'cancelled', got %q", errCancelled.Error())
}
})
}
+38 -11
View File
@@ -17,6 +17,7 @@ import (
"github.com/ollama/ollama/api"
"github.com/ollama/ollama/envconfig"
"github.com/ollama/ollama/internal/modelref"
"github.com/ollama/ollama/readline"
"github.com/ollama/ollama/types/errtypes"
"github.com/ollama/ollama/types/model"
@@ -46,7 +47,7 @@ func generateInteractive(cmd *cobra.Command, opts runOptions) error {
fmt.Fprintln(os.Stderr, "Use \"\"\" to begin a multi-line message.")
if opts.MultiModal {
fmt.Fprintf(os.Stderr, "Use %s to include .jpg, .png, or .webp images.\n", filepath.FromSlash("/path/to/file"))
fmt.Fprintf(os.Stderr, "Use %s to include .jpg, .png, .webp images, or .wav audio files.\n", filepath.FromSlash("/path/to/file"))
}
fmt.Fprintln(os.Stderr, "")
@@ -213,10 +214,17 @@ func generateInteractive(cmd *cobra.Command, opts runOptions) error {
}
origOpts := opts.Copy()
client, err := api.ClientFromEnvironment()
if err != nil {
fmt.Println("error: couldn't connect to ollama server")
return err
}
opts.Model = args[1]
opts.Messages = []api.Message{}
opts.LoadedMessages = nil
fmt.Printf("Loading model '%s'\n", opts.Model)
opts.Think, err = inferThinkingOption(nil, &opts, thinkExplicitlySet)
info, err := client.Show(cmd.Context(), &api.ShowRequest{Model: opts.Model})
if err != nil {
if strings.Contains(err.Error(), "not found") {
fmt.Printf("Couldn't find model '%s'\n", opts.Model)
@@ -225,6 +233,11 @@ func generateInteractive(cmd *cobra.Command, opts runOptions) error {
}
return err
}
applyShowResponseToRunOptions(&opts, info)
opts.Think, err = inferThinkingOption(&info.Capabilities, &opts, thinkExplicitlySet)
if err != nil {
return err
}
if err := loadOrUnloadModel(cmd, &opts); err != nil {
if strings.Contains(err.Error(), "not found") {
fmt.Printf("Couldn't find model '%s'\n", opts.Model)
@@ -540,6 +553,13 @@ func NewCreateRequest(name string, opts runOptions) *api.CreateRequest {
parentModel = ""
}
// Preserve explicit cloud intent for sessions started with `:cloud`.
// Cloud model metadata can return a source-less parent_model (for example
// "qwen3.5"), which would otherwise make `/save` create a local derivative.
if modelref.HasExplicitCloudSource(opts.Model) && !modelref.HasExplicitCloudSource(parentModel) {
parentModel = ""
}
req := &api.CreateRequest{
Model: name,
From: cmp.Or(parentModel, opts.Model),
@@ -553,8 +573,10 @@ func NewCreateRequest(name string, opts runOptions) *api.CreateRequest {
req.Parameters = opts.Options
}
if len(opts.Messages) > 0 {
req.Messages = opts.Messages
messages := slices.Clone(opts.LoadedMessages)
messages = append(messages, opts.Messages...)
if len(messages) > 0 {
req.Messages = messages
}
return req
@@ -584,7 +606,7 @@ func extractFileNames(input string) []string {
// Regex to match file paths starting with optional drive letter, / ./ \ or .\ and include escaped or unescaped spaces (\ or %20)
// and followed by more characters and a file extension
// This will capture non filename strings, but we'll check for file existence to remove mismatches
regexPattern := `(?:[a-zA-Z]:)?(?:\./|/|\\)[\S\\ ]+?\.(?i:jpg|jpeg|png|webp)\b`
regexPattern := `(?:[a-zA-Z]:)?(?:\./|/|\\)[\S\\ ]+?\.(?i:jpg|jpeg|png|webp|wav)\b`
re := regexp.MustCompile(regexPattern)
return re.FindAllString(input, -1)
@@ -600,10 +622,16 @@ func extractFileData(input string) (string, []api.ImageData, error) {
if errors.Is(err, os.ErrNotExist) {
continue
} else if err != nil {
fmt.Fprintf(os.Stderr, "Couldn't process image: %q\n", err)
fmt.Fprintf(os.Stderr, "Couldn't process file: %q\n", err)
return "", imgs, err
}
fmt.Fprintf(os.Stderr, "Added image '%s'\n", nfp)
ext := strings.ToLower(filepath.Ext(nfp))
switch ext {
case ".wav":
fmt.Fprintf(os.Stderr, "Added audio '%s'\n", nfp)
default:
fmt.Fprintf(os.Stderr, "Added image '%s'\n", nfp)
}
input = strings.ReplaceAll(input, "'"+nfp+"'", "")
input = strings.ReplaceAll(input, "'"+fp+"'", "")
input = strings.ReplaceAll(input, fp, "")
@@ -677,9 +705,9 @@ func getImageData(filePath string) ([]byte, error) {
}
contentType := http.DetectContentType(buf)
allowedTypes := []string{"image/jpeg", "image/jpg", "image/png", "image/webp"}
allowedTypes := []string{"image/jpeg", "image/jpg", "image/png", "image/webp", "audio/wave"}
if !slices.Contains(allowedTypes, contentType) {
return nil, fmt.Errorf("invalid image type: %s", contentType)
return nil, fmt.Errorf("invalid file type: %s", contentType)
}
info, err := file.Stat()
@@ -687,8 +715,7 @@ func getImageData(filePath string) ([]byte, error) {
return nil, err
}
// Check if the file size exceeds 100MB
var maxSize int64 = 100 * 1024 * 1024 // 100MB in bytes
var maxSize int64 = 100 * 1024 * 1024 // 100MB
if info.Size() > maxSize {
return nil, errors.New("file size exceeds maximum limit (100MB)")
}
+30
View File
@@ -84,3 +84,33 @@ func TestExtractFileDataRemovesQuotedFilepath(t *testing.T) {
assert.Len(t, imgs, 1)
assert.Equal(t, cleaned, "before after")
}
func TestExtractFileDataWAV(t *testing.T) {
dir := t.TempDir()
fp := filepath.Join(dir, "sample.wav")
data := make([]byte, 600)
copy(data[:44], []byte{
'R', 'I', 'F', 'F',
0x58, 0x02, 0x00, 0x00, // file size - 8
'W', 'A', 'V', 'E',
'f', 'm', 't', ' ',
0x10, 0x00, 0x00, 0x00, // fmt chunk size
0x01, 0x00, // PCM
0x01, 0x00, // mono
0x80, 0x3e, 0x00, 0x00, // 16000 Hz
0x00, 0x7d, 0x00, 0x00, // byte rate
0x02, 0x00, // block align
0x10, 0x00, // 16-bit
'd', 'a', 't', 'a',
0x34, 0x02, 0x00, 0x00, // data size
})
if err := os.WriteFile(fp, data, 0o600); err != nil {
t.Fatalf("failed to write test audio: %v", err)
}
input := "before " + fp + " after"
cleaned, imgs, err := extractFileData(input)
assert.NoError(t, err)
assert.Len(t, imgs, 1)
assert.Equal(t, "before after", cleaned)
}
+176
View File
@@ -0,0 +1,176 @@
// Package fileutil provides small shared helpers for reading JSON files
// and writing config files with backup-on-overwrite semantics.
package fileutil
import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
)
// Keep a bounded number of backups per file so config backups do not grow
// without limit. We keep the 5 most recent backups and do not pin the oldest.
const maxBackupsPerFile = 5
// ReadJSON reads a JSON object file into a generic map.
func ReadJSON(path string) (map[string]any, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var result map[string]any
if err := json.Unmarshal(data, &result); err != nil {
return nil, err
}
return result, nil
}
func copyFile(src, dst string) error {
info, err := os.Stat(src)
if err != nil {
return err
}
data, err := os.ReadFile(src)
if err != nil {
return err
}
return os.WriteFile(dst, data, info.Mode().Perm())
}
// BackupDir returns the shared backup root used before overwriting files.
func BackupDir() string {
if home, err := os.UserHomeDir(); err == nil && home != "" {
return filepath.Join(home, ".ollama", "backup")
}
return filepath.Join(os.TempDir(), "ollama-backup")
}
func writeBackupCopy(srcPath string, integration string) (string, error) {
dir := BackupDir()
name := filepath.Base(srcPath)
if integration != "" {
dir = filepath.Join(dir, integration)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
backupPath := filepath.Join(dir, fmt.Sprintf("%s.%d", name, time.Now().Unix()))
if err := copyFile(srcPath, backupPath); err != nil {
return "", err
}
pruneOldBackups(dir, name, maxBackupsPerFile)
return backupPath, nil
}
// WriteWithBackup writes data to path via temp file + rename, backing up any
// existing file first. Callers may optionally pass one integration name to
// store backups under BackupDir()/.../<integration>/.
func WriteWithBackup(path string, data []byte, integration ...string) error {
backupIntegration := ""
if len(integration) > 0 {
backupIntegration = integration[0]
}
var backupPath string
// backup must be created before any writes to the target file
if existingContent, err := os.ReadFile(path); err == nil {
if bytes.Equal(existingContent, data) {
return nil
}
backupPath, err = writeBackupCopy(path, backupIntegration)
if err != nil {
return fmt.Errorf("backup failed: %w", err)
}
} else if !os.IsNotExist(err) {
return fmt.Errorf("read existing file: %w", err)
}
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, ".tmp-*")
if err != nil {
return fmt.Errorf("create temp failed: %w", err)
}
tmpPath := tmp.Name()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("write failed: %w", err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
_ = os.Remove(tmpPath)
return fmt.Errorf("sync failed: %w", err)
}
if err := tmp.Close(); err != nil {
_ = os.Remove(tmpPath)
return fmt.Errorf("close failed: %w", err)
}
if err := os.Rename(tmpPath, path); err != nil {
_ = os.Remove(tmpPath)
if backupPath != "" {
_ = copyFile(backupPath, path)
}
return fmt.Errorf("rename failed: %w", err)
}
return nil
}
func pruneOldBackups(dir, name string, keep int) {
if keep < 1 {
return
}
entries, err := os.ReadDir(dir)
if err != nil {
return
}
type backupEntry struct {
name string
timestamp int64
}
prefix := name + "."
backups := make([]backupEntry, 0, len(entries))
for _, entry := range entries {
if entry.IsDir() || !strings.HasPrefix(entry.Name(), prefix) {
continue
}
timestamp, err := strconv.ParseInt(strings.TrimPrefix(entry.Name(), prefix), 10, 64)
if err != nil {
continue
}
backups = append(backups, backupEntry{
name: entry.Name(),
timestamp: timestamp,
})
}
if len(backups) <= keep {
return
}
sort.Slice(backups, func(i, j int) bool {
if backups[i].timestamp != backups[j].timestamp {
return backups[i].timestamp > backups[j].timestamp
}
return backups[i].name > backups[j].name
})
for _, backup := range backups[keep:] {
_ = os.Remove(filepath.Join(dir, backup.name))
}
}
@@ -1,4 +1,4 @@
package config
package fileutil
import (
"encoding/json"
@@ -9,6 +9,27 @@ import (
"testing"
)
func TestMain(m *testing.M) {
tmpRoot, err := os.MkdirTemp("", "fileutil-test-*")
if err != nil {
panic(err)
}
if err := os.Setenv("TMPDIR", tmpRoot); err != nil {
panic(err)
}
if err := os.Setenv("HOME", tmpRoot); err != nil {
panic(err)
}
if err := os.Setenv("USERPROFILE", tmpRoot); err != nil {
panic(err)
}
code := m.Run()
_ = os.RemoveAll(tmpRoot)
os.Exit(code)
}
func mustMarshal(t *testing.T, v any) []byte {
t.Helper()
data, err := json.MarshalIndent(v, "", " ")
@@ -18,14 +39,30 @@ func mustMarshal(t *testing.T, v any) []byte {
return data
}
func isolatedTempDir(t *testing.T) string {
t.Helper()
return t.TempDir()
}
func TestWriteWithBackup(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
t.Run("uses ollama directory under home", func(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
want := filepath.Join(home, ".ollama", "backup")
if got := BackupDir(); got != want {
t.Fatalf("BackupDir() = %q, want %q", got, want)
}
})
t.Run("creates file", func(t *testing.T) {
path := filepath.Join(tmpDir, "new.json")
data := mustMarshal(t, map[string]string{"key": "value"})
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
@@ -43,17 +80,17 @@ func TestWriteWithBackup(t *testing.T) {
}
})
t.Run("creates backup in /tmp/ollama-backups", func(t *testing.T) {
t.Run("creates backup in the shared backup directory", func(t *testing.T) {
path := filepath.Join(tmpDir, "backup.json")
os.WriteFile(path, []byte(`{"original": true}`), 0o644)
data := mustMarshal(t, map[string]bool{"updated": true})
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
entries, err := os.ReadDir(backupDir())
entries, err := os.ReadDir(BackupDir())
if err != nil {
t.Fatal("backup directory not created")
}
@@ -63,7 +100,7 @@ func TestWriteWithBackup(t *testing.T) {
if filepath.Ext(entry.Name()) != ".json" {
name := entry.Name()
if len(name) > len("backup.json.") && name[:len("backup.json.")] == "backup.json." {
backupPath := filepath.Join(backupDir(), name)
backupPath := filepath.Join(BackupDir(), name)
backup, err := os.ReadFile(backupPath)
if err == nil {
var backupData map[string]bool
@@ -79,7 +116,7 @@ func TestWriteWithBackup(t *testing.T) {
}
if !foundBackup {
t.Error("backup file not created in /tmp/ollama-backups")
t.Error("backup file not created in backup directory")
}
current, _ := os.ReadFile(path)
@@ -90,15 +127,44 @@ func TestWriteWithBackup(t *testing.T) {
}
})
t.Run("stores hinted backups under a subdirectory", func(t *testing.T) {
path := filepath.Join(tmpDir, "hinted.json")
os.WriteFile(path, []byte(`{"original": true}`), 0o644)
data := mustMarshal(t, map[string]bool{"updated": true})
if err := WriteWithBackup(path, data, "openclaw"); err != nil {
t.Fatal(err)
}
entries, err := os.ReadDir(filepath.Join(BackupDir(), "openclaw"))
if err != nil {
t.Fatal(err)
}
var found bool
for _, entry := range entries {
name := entry.Name()
if len(name) > len("hinted.json.") && name[:len("hinted.json.")] == "hinted.json." {
found = true
_ = os.Remove(filepath.Join(BackupDir(), "openclaw", name))
break
}
}
if !found {
t.Error("backup file was not created under hint directory")
}
})
t.Run("no backup for new file", func(t *testing.T) {
path := filepath.Join(tmpDir, "nobak.json")
data := mustMarshal(t, map[string]string{"new": "file"})
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
entries, _ := os.ReadDir(backupDir())
entries, _ := os.ReadDir(BackupDir())
for _, entry := range entries {
if len(entry.Name()) > len("nobak.json.") && entry.Name()[:len("nobak.json.")] == "nobak.json." {
t.Error("backup should not exist for new file")
@@ -111,11 +177,11 @@ func TestWriteWithBackup(t *testing.T) {
data := mustMarshal(t, map[string]string{"key": "value"})
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
entries1, _ := os.ReadDir(backupDir())
entries1, _ := os.ReadDir(BackupDir())
countBefore := 0
for _, e := range entries1 {
if len(e.Name()) > len("unchanged.json.") && e.Name()[:len("unchanged.json.")] == "unchanged.json." {
@@ -123,11 +189,11 @@ func TestWriteWithBackup(t *testing.T) {
}
}
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
entries2, _ := os.ReadDir(backupDir())
entries2, _ := os.ReadDir(BackupDir())
countAfter := 0
for _, e := range entries2 {
if len(e.Name()) > len("unchanged.json.") && e.Name()[:len("unchanged.json.")] == "unchanged.json." {
@@ -145,11 +211,11 @@ func TestWriteWithBackup(t *testing.T) {
os.WriteFile(path, []byte(`{"v": 1}`), 0o644)
data := mustMarshal(t, map[string]int{"v": 2})
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatal(err)
}
entries, _ := os.ReadDir(backupDir())
entries, _ := os.ReadDir(BackupDir())
var found bool
for _, entry := range entries {
name := entry.Name()
@@ -161,7 +227,7 @@ func TestWriteWithBackup(t *testing.T) {
}
}
found = true
os.Remove(filepath.Join(backupDir(), name))
os.Remove(filepath.Join(BackupDir(), name))
break
}
}
@@ -169,6 +235,35 @@ func TestWriteWithBackup(t *testing.T) {
t.Error("backup file with timestamp not found")
}
})
t.Run("retains only the five newest backups per file", func(t *testing.T) {
path := filepath.Join(tmpDir, "pruned.json")
if err := os.WriteFile(path, []byte(`{"v": 0}`), 0o644); err != nil {
t.Fatal(err)
}
for i := 1; i <= maxBackupsPerFile; i++ {
backupPath := filepath.Join(BackupDir(), fmt.Sprintf("pruned.json.%d", i))
if err := os.WriteFile(backupPath, []byte(fmt.Sprintf(`{"v": %d}`, i)), 0o644); err != nil {
t.Fatal(err)
}
}
if err := WriteWithBackup(path, []byte(`{"v": 1}`)); err != nil {
t.Fatal(err)
}
backups, err := filepath.Glob(filepath.Join(BackupDir(), "pruned.json.*"))
if err != nil {
t.Fatal(err)
}
if len(backups) != maxBackupsPerFile {
t.Fatalf("expected %d backups after pruning, got %d", maxBackupsPerFile, len(backups))
}
if _, err := os.Stat(filepath.Join(BackupDir(), "pruned.json.1")); !os.IsNotExist(err) {
t.Fatalf("expected oldest backup to be pruned, stat err = %v", err)
}
})
}
// Edge case tests for files.go
@@ -180,7 +275,7 @@ func TestWriteWithBackup_FailsIfBackupFails(t *testing.T) {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "config.json")
// Create original file
@@ -188,13 +283,13 @@ func TestWriteWithBackup_FailsIfBackupFails(t *testing.T) {
os.WriteFile(path, originalContent, 0o644)
// Make backup directory read-only to force backup failure
backupDir := backupDir()
backupDir := BackupDir()
os.MkdirAll(backupDir, 0o755)
os.Chmod(backupDir, 0o444) // Read-only
defer os.Chmod(backupDir, 0o755)
newContent := []byte(`{"updated": true}`)
err := writeWithBackup(path, newContent)
err := WriteWithBackup(path, newContent)
// Should fail because backup couldn't be created
if err == nil {
@@ -215,7 +310,7 @@ func TestWriteWithBackup_PermissionDenied(t *testing.T) {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
// Create a read-only directory
readOnlyDir := filepath.Join(tmpDir, "readonly")
@@ -224,20 +319,50 @@ func TestWriteWithBackup_PermissionDenied(t *testing.T) {
defer os.Chmod(readOnlyDir, 0o755)
path := filepath.Join(readOnlyDir, "config.json")
err := writeWithBackup(path, []byte(`{"test": true}`))
err := WriteWithBackup(path, []byte(`{"test": true}`))
if err == nil {
t.Error("expected permission error, got nil")
}
}
func TestWriteWithBackup_UnchangedContentIsNoOp(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "unchanged-noop.json")
data := []byte(`{"same":true}`)
if err := os.WriteFile(path, data, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chmod(tmpDir, 0o555); err != nil {
t.Fatal(err)
}
defer os.Chmod(tmpDir, 0o755)
if err := WriteWithBackup(path, data); err != nil {
t.Fatalf("expected unchanged write to be a no-op, got %v", err)
}
backups, err := filepath.Glob(filepath.Join(BackupDir(), "unchanged-noop.json.*"))
if err != nil {
t.Fatal(err)
}
if len(backups) != 0 {
t.Fatalf("expected no backups for unchanged content, got %d", len(backups))
}
}
// TestWriteWithBackup_DirectoryDoesNotExist verifies behavior when target directory doesn't exist.
// writeWithBackup doesn't create directories - caller is responsible.
func TestWriteWithBackup_DirectoryDoesNotExist(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "nonexistent", "subdir", "config.json")
err := writeWithBackup(path, []byte(`{"test": true}`))
err := WriteWithBackup(path, []byte(`{"test": true}`))
// Should fail because directory doesn't exist
if err == nil {
@@ -252,7 +377,7 @@ func TestWriteWithBackup_SymlinkTarget(t *testing.T) {
t.Skip("symlink tests may require admin on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
realFile := filepath.Join(tmpDir, "real.json")
symlink := filepath.Join(tmpDir, "link.json")
@@ -261,7 +386,7 @@ func TestWriteWithBackup_SymlinkTarget(t *testing.T) {
os.Symlink(realFile, symlink)
// Write through symlink
err := writeWithBackup(symlink, []byte(`{"v": 2}`))
err := WriteWithBackup(symlink, []byte(`{"v": 2}`))
if err != nil {
t.Fatalf("writeWithBackup through symlink failed: %v", err)
}
@@ -276,15 +401,15 @@ func TestWriteWithBackup_SymlinkTarget(t *testing.T) {
// TestBackupToTmp_SpecialCharsInFilename verifies backup works with special characters.
// User may have config files with unusual names.
func TestBackupToTmp_SpecialCharsInFilename(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
// File with spaces and special chars
path := filepath.Join(tmpDir, "my config (backup).json")
os.WriteFile(path, []byte(`{"test": true}`), 0o644)
backupPath, err := backupToTmp(path)
backupPath, err := writeBackupCopy(path, "")
if err != nil {
t.Fatalf("backupToTmp with special chars failed: %v", err)
t.Fatalf("writeBackupCopy with special chars failed: %v", err)
}
// Verify backup exists and has correct content
@@ -305,7 +430,7 @@ func TestCopyFile_PreservesPermissions(t *testing.T) {
t.Skip("permission preservation tests unreliable on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
src := filepath.Join(tmpDir, "src.json")
dst := filepath.Join(tmpDir, "dst.json")
@@ -327,7 +452,7 @@ func TestCopyFile_PreservesPermissions(t *testing.T) {
// TestCopyFile_SourceNotFound verifies clear error when source doesn't exist.
func TestCopyFile_SourceNotFound(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
src := filepath.Join(tmpDir, "nonexistent.json")
dst := filepath.Join(tmpDir, "dst.json")
@@ -339,11 +464,11 @@ func TestCopyFile_SourceNotFound(t *testing.T) {
// TestWriteWithBackup_TargetIsDirectory verifies error when path points to a directory.
func TestWriteWithBackup_TargetIsDirectory(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
dirPath := filepath.Join(tmpDir, "actualdir")
os.MkdirAll(dirPath, 0o755)
err := writeWithBackup(dirPath, []byte(`{"test": true}`))
err := WriteWithBackup(dirPath, []byte(`{"test": true}`))
if err == nil {
t.Error("expected error when target is a directory, got nil")
}
@@ -351,10 +476,10 @@ func TestWriteWithBackup_TargetIsDirectory(t *testing.T) {
// TestWriteWithBackup_EmptyData verifies writing zero bytes works correctly.
func TestWriteWithBackup_EmptyData(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "empty.json")
err := writeWithBackup(path, []byte{})
err := WriteWithBackup(path, []byte{})
if err != nil {
t.Fatalf("writeWithBackup with empty data failed: %v", err)
}
@@ -375,7 +500,7 @@ func TestWriteWithBackup_FileUnreadableButDirWritable(t *testing.T) {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "unreadable.json")
// Create file and make it unreadable
@@ -384,7 +509,7 @@ func TestWriteWithBackup_FileUnreadableButDirWritable(t *testing.T) {
defer os.Chmod(path, 0o644)
// Should fail because we can't read the file to compare/backup
err := writeWithBackup(path, []byte(`{"updated": true}`))
err := WriteWithBackup(path, []byte(`{"updated": true}`))
if err == nil {
t.Error("expected error when file is unreadable, got nil")
}
@@ -393,7 +518,7 @@ func TestWriteWithBackup_FileUnreadableButDirWritable(t *testing.T) {
// TestWriteWithBackup_RapidSuccessiveWrites verifies backup works with multiple writes
// within the same second (timestamp collision scenario).
func TestWriteWithBackup_RapidSuccessiveWrites(t *testing.T) {
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
path := filepath.Join(tmpDir, "rapid.json")
// Create initial file
@@ -402,7 +527,7 @@ func TestWriteWithBackup_RapidSuccessiveWrites(t *testing.T) {
// Rapid successive writes
for i := 1; i <= 3; i++ {
data := []byte(fmt.Sprintf(`{"v": %d}`, i))
if err := writeWithBackup(path, data); err != nil {
if err := WriteWithBackup(path, data); err != nil {
t.Fatalf("write %d failed: %v", i, err)
}
}
@@ -414,7 +539,7 @@ func TestWriteWithBackup_RapidSuccessiveWrites(t *testing.T) {
}
// Verify at least one backup exists
entries, _ := os.ReadDir(backupDir())
entries, _ := os.ReadDir(BackupDir())
var backupCount int
for _, e := range entries {
if len(e.Name()) > len("rapid.json.") && e.Name()[:len("rapid.json.")] == "rapid.json." {
@@ -432,8 +557,9 @@ func TestWriteWithBackup_BackupDirIsFile(t *testing.T) {
t.Skip("test modifies system temp directory")
}
tmpDir := isolatedTempDir(t)
// Create a file at the backup directory path
backupPath := backupDir()
backupPath := BackupDir()
// Clean up any existing directory first
os.RemoveAll(backupPath)
// Create a file instead of directory
@@ -443,11 +569,10 @@ func TestWriteWithBackup_BackupDirIsFile(t *testing.T) {
os.MkdirAll(backupPath, 0o755)
}()
tmpDir := t.TempDir()
path := filepath.Join(tmpDir, "test.json")
os.WriteFile(path, []byte(`{"original": true}`), 0o644)
err := writeWithBackup(path, []byte(`{"updated": true}`))
err := WriteWithBackup(path, []byte(`{"updated": true}`))
if err == nil {
t.Error("expected error when backup dir is a file, got nil")
}
@@ -459,7 +584,7 @@ func TestWriteWithBackup_NoOrphanTempFiles(t *testing.T) {
t.Skip("permission tests unreliable on Windows")
}
tmpDir := t.TempDir()
tmpDir := isolatedTempDir(t)
// Count existing temp files
countTempFiles := func() int {
@@ -493,7 +618,7 @@ func TestWriteWithBackup_NoOrphanTempFiles(t *testing.T) {
badPath := filepath.Join(tmpDir, "isdir")
os.MkdirAll(badPath, 0o755)
_ = writeWithBackup(badPath, []byte(`{"test": true}`))
_ = WriteWithBackup(badPath, []byte(`{"test": true}`))
after := countTempFiles()
if after > before {
+371
View File
@@ -0,0 +1,371 @@
package launch
import (
"context"
"errors"
"fmt"
"net/http"
"os"
"strings"
"time"
"github.com/ollama/ollama/api"
)
const (
// DefaultUpgradeURL is the fixed destination for subscription upgrades.
DefaultUpgradeURL = "https://ollama.com/upgrade"
accountCheckTimeout = 3 * time.Second
)
var (
ErrPlanVerificationUnavailable = errors.New("Could not verify your plan. Try again in a moment.")
errUpgradeCancelled = errors.New("upgrade cancelled")
)
type accountStateStatus int
const (
accountStateUnknown accountStateStatus = iota
accountStateSignedOut
accountStateSignedIn
)
type AccountState struct {
Status accountStateStatus
Plan string
}
type AccountStatePrefetch struct {
done chan struct{}
state AccountState
}
func StartAccountStatePrefetch(ctx context.Context) *AccountStatePrefetch {
if ctx == nil {
ctx = context.Background()
}
p := &AccountStatePrefetch{done: make(chan struct{})}
go func() {
state := AccountState{Status: accountStateUnknown}
client, err := api.ClientFromEnvironment()
if err == nil {
prefetchCtx, cancel := context.WithTimeout(ctx, accountCheckTimeout)
defer cancel()
if disabled, known := cloudStatusDisabled(prefetchCtx, client); !known || !disabled {
state = launchAccountState(prefetchCtx, client)
}
}
p.state = state
close(p.done)
}()
return p
}
func (p *AccountStatePrefetch) StateIfReady() *AccountState {
if p == nil {
return nil
}
select {
case <-p.done:
state := p.state
return &state
default:
return nil
}
}
func (p *AccountStatePrefetch) StateUpdates(ctx context.Context) <-chan *AccountState {
if p == nil {
return nil
}
if ctx == nil {
ctx = context.Background()
}
out := make(chan *AccountState, 1)
go func() {
defer close(out)
select {
case <-p.done:
if p.state.Status == accountStateUnknown {
return
}
state := p.state
select {
case out <- &state:
case <-ctx.Done():
}
case <-ctx.Done():
}
}()
return out
}
func launchAccountState(ctx context.Context, client *api.Client) AccountState {
if client == nil {
return AccountState{Status: accountStateUnknown}
}
user, err := whoamiWithTimeout(ctx, client)
if err != nil {
var authErr api.AuthorizationError
if errors.As(err, &authErr) && authErr.StatusCode == http.StatusUnauthorized {
return AccountState{Status: accountStateSignedOut}
}
return AccountState{Status: accountStateUnknown}
}
if user == nil || strings.TrimSpace(user.Name) == "" {
return AccountState{Status: accountStateSignedOut}
}
return AccountState{
Status: accountStateSignedIn,
Plan: strings.TrimSpace(user.Plan),
}
}
func whoamiWithTimeout(ctx context.Context, client *api.Client) (*api.UserResponse, error) {
if ctx == nil {
ctx = context.Background()
}
checkCtx, cancel := context.WithTimeout(ctx, accountCheckTimeout)
defer cancel()
return client.Whoami(checkCtx)
}
func ApplyAccountStateToSelectionItems(items []ModelItem, state AccountState) []SelectionItem {
out := make([]SelectionItem, len(items))
for i, item := range items {
out[i] = SelectionItem{
Name: item.Name,
Description: item.Description,
Recommended: item.Recommended,
AvailabilityBadge: availabilityBadge(item, state),
}
}
return out
}
func SelectionItemsWithAccountState(items []ModelItem, state *AccountState) []SelectionItem {
if state == nil || !selectionItemsNeedAccountState(items) {
return ApplyAccountStateToSelectionItems(items, AccountState{Status: accountStateUnknown})
}
return ApplyAccountStateToSelectionItems(items, *state)
}
func selectionItemsNeedAccountState(items []ModelItem) bool {
for _, item := range items {
if isCloudModelName(item.Name) && itemHasRecommendationMetadata(item) {
return true
}
}
return false
}
func (c *launcherClient) selectionItemUpdates(ctx context.Context, items []ModelItem, state *AccountState) <-chan []SelectionItem {
if !selectionItemsNeedAccountState(items) || state != nil {
return nil
}
if ctx == nil {
ctx = context.Background()
}
stateUpdates := c.accountStateUpdateSource(ctx)
if stateUpdates == nil {
return nil
}
out := make(chan []SelectionItem, 1)
go func() {
defer close(out)
select {
case state, ok := <-stateUpdates:
if !ok || state == nil {
return
}
select {
case out <- SelectionItemsWithAccountState(items, state):
case <-ctx.Done():
}
case <-ctx.Done():
}
}()
return out
}
func (c *launcherClient) accountStateUpdateSource(ctx context.Context) <-chan *AccountState {
if c.accountStateUpdates != nil {
return c.accountStateUpdates(ctx)
}
if c.apiClient == nil {
return nil
}
out := make(chan *AccountState, 1)
go func() {
defer close(out)
state := launchAccountState(ctx, c.apiClient)
if state.Status == accountStateUnknown {
return
}
select {
case out <- &state:
case <-ctx.Done():
}
}()
return out
}
func availabilityBadge(item ModelItem, state AccountState) string {
if !isCloudModelName(item.Name) {
return ""
}
switch state.Status {
case accountStateSignedOut:
if itemHasRecommendationMetadata(item) {
return "Sign in required"
}
case accountStateSignedIn:
if item.RequiredPlan != "" && !PlanSatisfies(state.Plan, item.RequiredPlan) {
return "Upgrade required"
}
}
return ""
}
func itemHasRecommendationMetadata(item ModelItem) bool {
return item.Recommended || strings.TrimSpace(item.RequiredPlan) != ""
}
func (c *launcherClient) ensureCloudModelAccess(ctx context.Context, model string) error {
item, ok := c.modelRecommendationItem(ctx, model)
if !ok || strings.TrimSpace(item.RequiredPlan) == "" {
return nil
}
state := launchAccountState(ctx, c.apiClient)
if state.Status != accountStateUnknown {
c.accountState = &state
}
if state.Status == accountStateUnknown {
return ErrPlanVerificationUnavailable
}
if state.Status == accountStateSignedOut {
if err := ensureCloudAuth(ctx, c.apiClient, model); err != nil {
return err
}
state = launchAccountState(ctx, c.apiClient)
if state.Status != accountStateUnknown {
c.accountState = &state
}
if state.Status == accountStateUnknown {
return ErrPlanVerificationUnavailable
}
}
if PlanSatisfies(state.Plan, item.RequiredPlan) {
return nil
}
if err := c.runUpgradeFlow(ctx, item); err != nil {
return err
}
state = launchAccountState(ctx, c.apiClient)
if state.Status == accountStateUnknown {
return ErrPlanVerificationUnavailable
}
if state.Status != accountStateSignedIn || !PlanSatisfies(state.Plan, item.RequiredPlan) {
return errUpgradeCancelled
}
return nil
}
func (c *launcherClient) modelRecommendationItem(ctx context.Context, model string) (ModelItem, bool) {
for _, item := range c.recommendations(ctx) {
if item.Name == model {
return item, true
}
}
return ModelItem{}, false
}
func (c *launcherClient) runUpgradeFlow(ctx context.Context, item ModelItem) error {
if DefaultUpgrade != nil {
if _, err := DefaultUpgrade(item.Name, item.RequiredPlan); err != nil {
if errors.Is(err, ErrCancelled) {
return errUpgradeCancelled
}
return err
}
return nil
}
yes, err := ConfirmPrompt(fmt.Sprintf("Upgrade to use %s?", item.Name))
if errors.Is(err, ErrCancelled) {
return errUpgradeCancelled
}
if err != nil {
return err
}
if !yes {
return errUpgradeCancelled
}
fmt.Fprintf(os.Stderr, "\nTo upgrade, navigate to:\n %s\n\n", DefaultUpgradeURL)
openNow, err := ConfirmPrompt("Open now?")
if errors.Is(err, ErrCancelled) {
return errUpgradeCancelled
}
if err != nil {
return err
}
if openNow {
OpenBrowser(DefaultUpgradeURL)
} else {
return errUpgradeCancelled
}
spinnerFrames := []string{"|", "/", "-", "\\"}
frame := 0
fmt.Fprintf(os.Stderr, "\033[90mwaiting for upgrade to complete... %s\033[0m", spinnerFrames[0])
ticker := time.NewTicker(200 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
fmt.Fprintf(os.Stderr, "\r\033[K")
return ctx.Err()
case <-ticker.C:
frame++
fmt.Fprintf(os.Stderr, "\r\033[90mwaiting for upgrade to complete... %s\033[0m", spinnerFrames[frame%len(spinnerFrames)])
if frame%10 != 0 {
continue
}
state := launchAccountState(ctx, c.apiClient)
if state.Status == accountStateUnknown {
fmt.Fprintf(os.Stderr, "\r\033[K")
return ErrPlanVerificationUnavailable
}
if state.Status == accountStateSignedIn && PlanSatisfies(state.Plan, item.RequiredPlan) {
fmt.Fprintf(os.Stderr, "\r\033[K\033[A\r\033[K\033[1mplan updated\033[0m\n")
return nil
}
}
}
}
// PlanSatisfies reports whether currentPlan can use a model that has a requiredPlan.
func PlanSatisfies(currentPlan, requiredPlan string) bool {
required := normalizePlan(requiredPlan)
if required == "" || required == "free" {
return true
}
current := normalizePlan(currentPlan)
return current != "" && current != "free"
}
func normalizePlan(plan string) string {
return strings.ToLower(strings.TrimSpace(plan))
}
+87
View File
@@ -0,0 +1,87 @@
package launch
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strconv"
"github.com/ollama/ollama/envconfig"
)
// Claude implements Runner for Claude Code integration.
type Claude struct{}
func (c *Claude) String() string { return "Claude Code" }
func (c *Claude) args(model string, extra []string) []string {
var args []string
if model != "" {
args = append(args, "--model", model)
}
args = append(args, extra...)
return args
}
func (c *Claude) findPath() (string, error) {
if p, err := exec.LookPath("claude"); err == nil {
return p, nil
}
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
name := "claude"
if runtime.GOOS == "windows" {
name = "claude.exe"
}
fallback := filepath.Join(home, ".claude", "local", name)
if _, err := os.Stat(fallback); err != nil {
return "", err
}
return fallback, nil
}
func (c *Claude) Run(model string, args []string) error {
claudePath, err := c.findPath()
if err != nil {
return fmt.Errorf("claude is not installed, install from https://code.claude.com/docs/en/quickstart")
}
cmd := exec.Command(claudePath, c.args(model, args)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
env := append(os.Environ(),
"ANTHROPIC_BASE_URL="+envconfig.Host().String(),
"ANTHROPIC_API_KEY=",
"ANTHROPIC_AUTH_TOKEN=ollama",
"CLAUDE_CODE_ATTRIBUTION_HEADER=0",
)
env = append(env, c.modelEnvVars(model)...)
cmd.Env = env
return cmd.Run()
}
// modelEnvVars returns Claude Code env vars that route all model tiers through Ollama.
func (c *Claude) modelEnvVars(model string) []string {
env := []string{
"ANTHROPIC_DEFAULT_OPUS_MODEL=" + model,
"ANTHROPIC_DEFAULT_SONNET_MODEL=" + model,
"ANTHROPIC_DEFAULT_HAIKU_MODEL=" + model,
"CLAUDE_CODE_SUBAGENT_MODEL=" + model,
}
if isCloudModelName(model) {
if l, ok := lookupCloudModelLimit(model); ok {
env = append(env, "CLAUDE_CODE_AUTO_COMPACT_WINDOW="+strconv.Itoa(l.Context))
}
}
return env
}
+889
View File
@@ -0,0 +1,889 @@
package launch
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/ollama/ollama/cmd/config"
"github.com/ollama/ollama/cmd/internal/fileutil"
"golang.org/x/term"
)
const (
claudeDesktopIntegrationName = "claude-desktop"
claudeDesktopProfileName = "Ollama"
claudeDesktopProfileID = "00000000-0000-4000-8000-000000000114"
claudeDesktopGatewayBaseURL = "https://ollama.com"
claudeDesktopAPIKeyURL = "https://ollama.com/settings/keys"
claudeDesktopModelLabel = "Ollama Cloud"
claudeDesktopSuccessMessage = "Claude Desktop profile changed to Ollama Cloud."
claudeDesktopRestoreMessage = "To restore the usual Claude profile, run: ollama launch claude-desktop --restore"
claudeDesktopRestoredMessage = "Claude Desktop restored to the usual Claude profile."
)
var (
claudeDesktopGOOS = runtime.GOOS
claudeDesktopUserHome = os.UserHomeDir
claudeDesktopStat = os.Stat
claudeDesktopOpenApp = defaultClaudeDesktopOpenApp
claudeDesktopOpenAppPath = defaultClaudeDesktopOpenAppPath
claudeDesktopQuitApp = defaultClaudeDesktopQuitApp
claudeDesktopIsRunning = defaultClaudeDesktopIsRunning
claudeDesktopRunningAppPath = defaultClaudeDesktopRunningAppPath
claudeDesktopGlob = filepath.Glob
claudeDesktopSleep = time.Sleep
claudeDesktopHTTPClient = http.DefaultClient
claudeDesktopPromptAPIKey = promptClaudeDesktopAPIKey
claudeDesktopValidateAPIKey = validateClaudeDesktopAPIKey
)
// ClaudeDesktop configures and launches Claude Desktop in third-party
// inference mode using Ollama Cloud as the gateway.
type ClaudeDesktop struct{}
func (c *ClaudeDesktop) String() string { return "Claude Desktop" }
func (c *ClaudeDesktop) Supported() error { return claudeDesktopSupported() }
func (c *ClaudeDesktop) Paths() []string {
return nil
}
func (c *ClaudeDesktop) AutodiscoveredModel() string {
return claudeDesktopModelLabel
}
func (c *ClaudeDesktop) ConfigureAutodiscovery() error {
if err := claudeDesktopSupported(); err != nil {
return err
}
targets, err := claudeDesktopTargetPaths()
if err != nil {
return err
}
key, err := claudeDesktopValidatedAPIKey(context.Background(), claudeDesktopTargetProfilePaths(targets))
if err != nil {
return err
}
for _, path := range targets.normalConfigs {
if err := writeClaudeDesktopDeploymentMode(path, "3p"); err != nil {
return err
}
}
for _, target := range targets.thirdPartyProfiles {
if err := writeClaudeDesktopDeploymentMode(target.desktopConfig, "3p"); err != nil {
return err
}
if err := writeClaudeDesktopMeta(target.meta, claudeDesktopProfileID, claudeDesktopProfileName); err != nil {
return err
}
if err := writeClaudeDesktopGatewayProfile(target.profile, key, true); err != nil {
return err
}
}
return nil
}
func (c *ClaudeDesktop) RestoreHint() string {
return claudeDesktopRestoreMessage
}
func (c *ClaudeDesktop) ConfigurationSuccessMessage() string {
return claudeDesktopSuccessMessage + "\n" + claudeDesktopRestoreMessage
}
func (c *ClaudeDesktop) RestoreSuccessMessage() string {
return claudeDesktopRestoredMessage
}
func (c *ClaudeDesktop) AutodiscoveryConfigured() bool {
targets, err := claudeDesktopTargetPaths()
if err != nil {
return false
}
return claudeDesktopTargetsConfigured(targets)
}
func (c *ClaudeDesktop) Onboard() error {
return config.MarkIntegrationOnboarded(claudeDesktopIntegrationName)
}
func (c *ClaudeDesktop) RequiresInteractiveOnboarding() bool {
return false
}
func (c *ClaudeDesktop) SkipModelReadiness() bool {
return true
}
func (c *ClaudeDesktop) Run(_ string, args []string) error {
if err := claudeDesktopSupported(); err != nil {
return err
}
if len(args) > 0 {
return fmt.Errorf("claude-desktop does not accept extra arguments")
}
return claudeDesktopLaunchOrRestart("Restart Claude Desktop to use Ollama?")
}
func (c *ClaudeDesktop) Restore() error {
if err := claudeDesktopSupported(); err != nil {
return err
}
targets, err := claudeDesktopTargetPaths()
if err != nil {
return err
}
for _, path := range targets.normalConfigs {
if err := writeClaudeDesktopDeploymentMode(path, "1p"); err != nil {
return err
}
}
for _, target := range targets.thirdPartyProfiles {
if err := writeClaudeDesktopDeploymentMode(target.desktopConfig, "1p"); err != nil {
return err
}
if err := restoreClaudeDesktopMeta(target.meta); err != nil {
return err
}
if err := restoreClaudeDesktopOllamaProfile(target.profile); err != nil {
return err
}
}
return claudeDesktopLaunchOrRestart("Restart Claude Desktop to use the usual Claude profile?")
}
func claudeDesktopSupported() error {
switch claudeDesktopGOOS {
case "darwin", "windows":
return nil
default:
return fmt.Errorf("Claude Desktop launch is only supported on macOS and Windows")
}
}
func claudeDesktopInstalled() bool {
if claudeDesktopAppPath() != "" {
return true
}
if claudeDesktopGOOS == "windows" && claudeDesktopIsRunning() {
return true
}
for _, dir := range claudeDesktopProfileDirCandidates(false) {
if _, err := claudeDesktopStat(dir); err == nil {
return true
}
}
return false
}
func claudeDesktopAppPath() string {
if claudeDesktopGOOS != "darwin" && claudeDesktopGOOS != "windows" {
return ""
}
for _, path := range claudeDesktopAppCandidates() {
if _, err := claudeDesktopStat(path); err == nil {
return path
}
}
return ""
}
func claudeDesktopAppCandidates() []string {
switch claudeDesktopGOOS {
case "darwin":
return claudeDesktopDarwinAppCandidates()
case "windows":
return claudeDesktopWindowsAppCandidates()
default:
return nil
}
}
func claudeDesktopDarwinAppCandidates() []string {
candidates := []string{"/Applications/Claude.app"}
if home, err := claudeDesktopUserHome(); err == nil {
candidates = append(candidates, filepath.Join(home, "Applications", "Claude.app"))
}
return candidates
}
func claudeDesktopWindowsAppCandidates() []string {
local, err := claudeDesktopLocalAppData()
if err != nil {
return nil
}
candidates := []string{
filepath.Join(local, "Programs", "Claude", "Claude.exe"),
filepath.Join(local, "Programs", "Claude Desktop", "Claude.exe"),
filepath.Join(local, "Claude", "Claude.exe"),
filepath.Join(local, "Claude Nest", "Claude.exe"),
filepath.Join(local, "Claude Desktop", "Claude.exe"),
filepath.Join(local, "AnthropicClaude", "Claude.exe"),
}
for _, pattern := range []string{
filepath.Join(local, "AnthropicClaude", "app-*", "Claude.exe"),
filepath.Join(local, "Programs", "Claude", "app-*", "Claude.exe"),
filepath.Join(local, "Programs", "Claude Desktop", "app-*", "Claude.exe"),
} {
matches, _ := claudeDesktopGlob(pattern)
candidates = append(candidates, matches...)
}
return claudeDesktopDedupePaths(candidates)
}
func claudeDesktopDedupePaths(paths []string) []string {
out := make([]string, 0, len(paths))
seen := make(map[string]bool, len(paths))
for _, path := range paths {
if strings.TrimSpace(path) == "" {
continue
}
key := strings.ToLower(path)
if seen[key] {
continue
}
seen[key] = true
out = append(out, path)
}
return out
}
type claudeDesktopPaths struct {
normalConfig string
desktopConfig string
meta string
profile string
}
type claudeDesktopThirdPartyPaths struct {
desktopConfig string
meta string
profile string
}
type claudeDesktopTargets struct {
normalConfigs []string
thirdPartyProfiles []claudeDesktopThirdPartyPaths
}
func claudeDesktopConfigPaths() (claudeDesktopPaths, error) {
switch claudeDesktopGOOS {
case "darwin":
return claudeDesktopDarwinConfigPaths()
case "windows":
return claudeDesktopWindowsConfigPaths()
default:
return claudeDesktopPaths{}, claudeDesktopSupported()
}
}
func claudeDesktopDarwinConfigPaths() (claudeDesktopPaths, error) {
normalRoots, thirdPartyRoots, err := claudeDesktopDarwinProfileRoots()
if err != nil {
return claudeDesktopPaths{}, err
}
normalBase := normalRoots[0]
thirdPartyBase := thirdPartyRoots[0]
return claudeDesktopPaths{
normalConfig: filepath.Join(normalBase, "claude_desktop_config.json"),
desktopConfig: filepath.Join(thirdPartyBase, "claude_desktop_config.json"),
meta: filepath.Join(thirdPartyBase, "configLibrary", "_meta.json"),
profile: filepath.Join(thirdPartyBase, "configLibrary", claudeDesktopProfileID+".json"),
}, nil
}
func claudeDesktopWindowsConfigPaths() (claudeDesktopPaths, error) {
normalBase, err := claudeDesktopProfileDir(true)
if err != nil {
return claudeDesktopPaths{}, err
}
thirdPartyBase, err := claudeDesktopProfileDir(false)
if err != nil {
return claudeDesktopPaths{}, err
}
return claudeDesktopPaths{
normalConfig: filepath.Join(normalBase, "claude_desktop_config.json"),
desktopConfig: filepath.Join(thirdPartyBase, "claude_desktop_config.json"),
meta: filepath.Join(thirdPartyBase, "configLibrary", "_meta.json"),
profile: filepath.Join(thirdPartyBase, "configLibrary", claudeDesktopProfileID+".json"),
}, nil
}
func claudeDesktopProfileDir(normal bool) (string, error) {
candidates := claudeDesktopProfileDirCandidates(normal)
if len(candidates) == 0 {
return "", fmt.Errorf("Claude Desktop profile directory could not be resolved")
}
for _, candidate := range candidates {
if _, err := claudeDesktopStat(candidate); err == nil {
return candidate, nil
}
}
return candidates[0], nil
}
func claudeDesktopProfileDirCandidates(normal bool) []string {
if claudeDesktopGOOS != "windows" {
return nil
}
normalRoots, thirdPartyRoots, err := claudeDesktopWindowsProfileRoots()
if err != nil {
return nil
}
if normal {
return normalRoots
}
return thirdPartyRoots
}
func claudeDesktopDarwinProfileRoots() ([]string, []string, error) {
home, err := claudeDesktopUserHome()
if err != nil {
return nil, nil, err
}
base := filepath.Join(home, "Library", "Application Support")
return []string{filepath.Join(base, "Claude")}, []string{filepath.Join(base, "Claude-3p")}, nil
}
func claudeDesktopWindowsProfileRoots() ([]string, []string, error) {
local, err := claudeDesktopLocalAppData()
if err != nil {
return nil, nil, err
}
normalRoots := []string{
filepath.Join(local, "Claude"),
filepath.Join(local, "Claude Nest"),
}
thirdPartyRoots := []string{
filepath.Join(local, "Claude-3p"),
filepath.Join(local, "Claude Nest-3p"),
}
return normalRoots, thirdPartyRoots, nil
}
func claudeDesktopTargetPaths() (claudeDesktopTargets, error) {
var (
normalRoots []string
thirdPartyRoots []string
err error
)
switch claudeDesktopGOOS {
case "darwin":
normalRoots, thirdPartyRoots, err = claudeDesktopDarwinProfileRoots()
case "windows":
normalRoots, thirdPartyRoots, err = claudeDesktopWindowsProfileRoots()
default:
err = claudeDesktopSupported()
}
if err != nil {
return claudeDesktopTargets{}, err
}
return newClaudeDesktopTargets(normalRoots, thirdPartyRoots), nil
}
func newClaudeDesktopTargets(normalRoots, thirdPartyRoots []string) claudeDesktopTargets {
targets := claudeDesktopTargets{}
for _, root := range claudeDesktopDedupePaths(normalRoots) {
targets.normalConfigs = append(targets.normalConfigs, filepath.Join(root, "claude_desktop_config.json"))
}
for _, root := range claudeDesktopDedupePaths(thirdPartyRoots) {
targets.thirdPartyProfiles = append(targets.thirdPartyProfiles, claudeDesktopThirdPartyPaths{
desktopConfig: filepath.Join(root, "claude_desktop_config.json"),
meta: filepath.Join(root, "configLibrary", "_meta.json"),
profile: filepath.Join(root, "configLibrary", claudeDesktopProfileID+".json"),
})
}
return targets
}
func claudeDesktopTargetProfilePaths(targets claudeDesktopTargets) []string {
paths := make([]string, 0, len(targets.thirdPartyProfiles))
for _, target := range targets.thirdPartyProfiles {
paths = append(paths, target.profile)
}
return paths
}
func claudeDesktopLocalAppData() (string, error) {
if local := strings.TrimSpace(os.Getenv("LOCALAPPDATA")); local != "" {
return local, nil
}
if home := strings.TrimSpace(os.Getenv("USERPROFILE")); home != "" {
return filepath.Join(home, "AppData", "Local"), nil
}
home, err := claudeDesktopUserHome()
if err != nil {
return "", err
}
return filepath.Join(home, "AppData", "Local"), nil
}
type claudeDesktopAPIKeySource int
const (
claudeDesktopAPIKeySourceNone claudeDesktopAPIKeySource = iota
claudeDesktopAPIKeySourceEnv
claudeDesktopAPIKeySourceProfile
)
func claudeDesktopValidatedAPIKey(ctx context.Context, profilePaths []string) (string, error) {
key, source, err := claudeDesktopAPIKey(profilePaths)
if err != nil {
return "", err
}
if err := claudeDesktopValidateAPIKey(ctx, key); err == nil {
return key, nil
} else if source != claudeDesktopAPIKeySourceProfile || !canPromptClaudeDesktopAPIKey() {
return "", err
}
return promptValidClaudeDesktopAPIKey(ctx)
}
func claudeDesktopAPIKey(profilePaths []string) (string, claudeDesktopAPIKeySource, error) {
if key := strings.TrimSpace(os.Getenv("OLLAMA_API_KEY")); key != "" {
return key, claudeDesktopAPIKeySourceEnv, nil
}
for _, profilePath := range profilePaths {
if key := readClaudeDesktopGatewayAPIKey(profilePath); key != "" {
return key, claudeDesktopAPIKeySourceProfile, nil
}
}
key, err := promptClaudeDesktopAPIKeyValue()
return key, claudeDesktopAPIKeySourceNone, err
}
func canPromptClaudeDesktopAPIKey() bool {
return isInteractiveSession() && !currentLaunchConfirmPolicy.requireYesMessage
}
func promptValidClaudeDesktopAPIKey(ctx context.Context) (string, error) {
key, err := promptClaudeDesktopAPIKeyValue()
if err != nil {
return "", err
}
if err := claudeDesktopValidateAPIKey(ctx, key); err != nil {
return "", err
}
return key, nil
}
func promptClaudeDesktopAPIKeyValue() (string, error) {
if !canPromptClaudeDesktopAPIKey() {
return "", missingClaudeDesktopAPIKeyError()
}
key, err := claudeDesktopPromptAPIKey()
if err != nil {
return "", err
}
key = strings.TrimSpace(key)
if key == "" {
return "", missingClaudeDesktopAPIKeyError()
}
return key, nil
}
func missingClaudeDesktopAPIKeyError() error {
return fmt.Errorf("OLLAMA_API_KEY is required for Claude Desktop. Create an API key at %s, then re-run with OLLAMA_API_KEY set", claudeDesktopAPIKeyURL)
}
func promptClaudeDesktopAPIKey() (string, error) {
fmt.Fprint(os.Stderr, claudeDesktopAPIKeyPrompt())
key, err := term.ReadPassword(int(os.Stdin.Fd()))
fmt.Fprintln(os.Stderr)
if err != nil {
return "", err
}
return string(key), nil
}
func claudeDesktopAPIKeyPrompt() string {
return fmt.Sprintf("Create an Ollama API key at %s\nEnter Ollama API key (input hidden): ", claudeDesktopAPIKeyURL)
}
func readClaudeDesktopGatewayAPIKey(path string) string {
cfg, err := readClaudeDesktopJSON(path)
if err != nil {
return ""
}
key, _ := cfg["inferenceGatewayApiKey"].(string)
return strings.TrimSpace(key)
}
func validateClaudeDesktopAPIKey(ctx context.Context, key string) error {
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
if claudeDesktopAPIKeyHasInvalidHeaderChars(key) {
return claudeDesktopAPIKeyVerificationError()
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, claudeDesktopGatewayBaseURL+"/v1/models", nil)
if err != nil {
return claudeDesktopAPIKeyVerificationError()
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Accept", "application/json")
resp, err := claudeDesktopHTTPClient.Do(req)
if err != nil {
return claudeDesktopAPIKeyVerificationError()
}
defer resp.Body.Close()
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 4<<10))
switch {
case resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden:
return fmt.Errorf("Ollama API key was rejected; create a valid key at %s", claudeDesktopAPIKeyURL)
case resp.StatusCode >= 200 && resp.StatusCode < 300:
return nil
default:
return fmt.Errorf("could not verify Ollama API key; ollama.com returned status %d, try again later", resp.StatusCode)
}
}
func claudeDesktopAPIKeyHasInvalidHeaderChars(key string) bool {
return strings.ContainsFunc(key, func(r rune) bool {
return r < ' ' || r == 0x7f
})
}
func claudeDesktopAPIKeyVerificationError() error {
return fmt.Errorf("could not verify Ollama API key; copy a key from %s and try again", claudeDesktopAPIKeyURL)
}
func writeClaudeDesktopDeploymentMode(path, mode string) error {
cfg, err := readClaudeDesktopJSONAllowMissing(path)
if err != nil {
return fmt.Errorf("parse Claude Desktop config: %w", err)
}
cfg["deploymentMode"] = mode
return writeClaudeDesktopJSON(path, cfg)
}
func writeClaudeDesktopMeta(path, id, name string) error {
meta, err := readClaudeDesktopJSONAllowMissing(path)
if err != nil {
return fmt.Errorf("parse Claude Desktop config metadata: %w", err)
}
meta["appliedId"] = id
entries := make([]any, 0)
for _, entry := range claudeDesktopAnySlice(meta["entries"]) {
entryMap, _ := entry.(map[string]any)
if entryMap == nil {
entries = append(entries, entry)
continue
}
if entryID, _ := entryMap["id"].(string); entryID == id {
continue
}
entries = append(entries, entryMap)
}
entries = append(entries, map[string]any{
"id": id,
"name": name,
})
meta["entries"] = entries
return writeClaudeDesktopJSON(path, meta)
}
func writeClaudeDesktopGatewayProfile(path string, apiKey string, forceChooser bool) error {
cfg, err := readClaudeDesktopJSONAllowMissing(path)
if err != nil {
return fmt.Errorf("parse Claude Desktop Ollama profile: %w", err)
}
cfg["inferenceProvider"] = "gateway"
cfg["inferenceGatewayBaseUrl"] = claudeDesktopGatewayBaseURL
cfg["inferenceGatewayApiKey"] = apiKey
cfg["inferenceGatewayAuthScheme"] = "bearer"
delete(cfg, "inferenceModels")
cfg["disableDeploymentModeChooser"] = forceChooser
return writeClaudeDesktopJSON(path, cfg)
}
func restoreClaudeDesktopMeta(path string) error {
meta, err := readClaudeDesktopJSONAllowMissing(path)
if err != nil {
return fmt.Errorf("parse Claude Desktop config metadata: %w", err)
}
if len(meta) == 0 {
return nil
}
changed := false
if appliedID, _ := meta["appliedId"].(string); appliedID == claudeDesktopProfileID {
delete(meta, "appliedId")
changed = true
}
entries := claudeDesktopAnySlice(meta["entries"])
if entries != nil {
filtered := make([]any, 0, len(entries))
for _, entry := range entries {
entryMap, _ := entry.(map[string]any)
if entryID, _ := entryMap["id"].(string); entryID == claudeDesktopProfileID {
changed = true
continue
}
filtered = append(filtered, entry)
}
meta["entries"] = filtered
}
if !changed {
return nil
}
return writeClaudeDesktopJSON(path, meta)
}
func restoreClaudeDesktopOllamaProfile(path string) error {
cfg, err := readClaudeDesktopJSONAllowMissing(path)
if err != nil {
return fmt.Errorf("parse Claude Desktop Ollama profile: %w", err)
}
if len(cfg) == 0 {
return nil
}
cfg["disableDeploymentModeChooser"] = false
delete(cfg, "inferenceProvider")
delete(cfg, "inferenceGatewayBaseUrl")
delete(cfg, "inferenceGatewayAuthScheme")
delete(cfg, "inferenceModels")
return writeClaudeDesktopJSON(path, cfg)
}
func readClaudeDesktopAppliedID(path string) string {
meta, err := readClaudeDesktopJSON(path)
if err != nil {
return ""
}
applied, _ := meta["appliedId"].(string)
return applied
}
func readClaudeDesktopDeploymentMode(path string) string {
cfg, err := readClaudeDesktopJSON(path)
if err != nil {
return ""
}
mode, _ := cfg["deploymentMode"].(string)
return mode
}
func claudeDesktopTargetsConfigured(targets claudeDesktopTargets) bool {
if len(targets.normalConfigs) == 0 || len(targets.thirdPartyProfiles) == 0 {
return false
}
for _, path := range targets.normalConfigs {
if readClaudeDesktopDeploymentMode(path) != "3p" {
return false
}
}
for _, target := range targets.thirdPartyProfiles {
if readClaudeDesktopDeploymentMode(target.desktopConfig) != "3p" {
return false
}
if !claudeDesktopThirdPartyProfileConfigured(target) {
return false
}
}
return true
}
func claudeDesktopThirdPartyProfileConfigured(target claudeDesktopThirdPartyPaths) bool {
if readClaudeDesktopAppliedID(target.meta) != claudeDesktopProfileID {
return false
}
cfg, err := readClaudeDesktopJSON(target.profile)
if err != nil {
return false
}
if s, _ := cfg["inferenceProvider"].(string); s != "gateway" {
return false
}
if s, _ := cfg["inferenceGatewayBaseUrl"].(string); strings.TrimRight(s, "/") != claudeDesktopGatewayBaseURL {
return false
}
if s, _ := cfg["inferenceGatewayApiKey"].(string); strings.TrimSpace(s) == "" {
return false
}
return true
}
func readClaudeDesktopJSONAllowMissing(path string) (map[string]any, error) {
cfg, err := readClaudeDesktopJSON(path)
if errors.Is(err, os.ErrNotExist) {
return map[string]any{}, nil
}
return cfg, err
}
func readClaudeDesktopJSON(path string) (map[string]any, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, err
}
if cfg == nil {
cfg = map[string]any{}
}
return cfg, nil
}
func writeClaudeDesktopJSON(path string, cfg any) error {
data, err := json.MarshalIndent(cfg, "", " ")
if err != nil {
return err
}
data = append(data, '\n')
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return fileutil.WriteWithBackup(path, data)
}
func claudeDesktopAnySlice(value any) []any {
switch v := value.(type) {
case []any:
return v
case nil:
return nil
default:
return nil
}
}
func claudeDesktopLaunchOrRestart(prompt string) error {
if !claudeDesktopIsRunning() {
return claudeDesktopOpenApp()
}
restartAppPath := ""
if claudeDesktopGOOS == "windows" {
restartAppPath = claudeDesktopRunningAppPath()
}
restart, err := ConfirmPrompt(prompt)
if err != nil {
return err
}
if !restart {
fmt.Fprintln(os.Stderr, "\nQuit and reopen Claude Desktop when you're ready for the profile change to take effect.")
return nil
}
if err := claudeDesktopQuitApp(); err != nil {
return fmt.Errorf("quit Claude Desktop: %w", err)
}
if err := waitForClaudeDesktopExit(30 * time.Second); err != nil {
return err
}
if restartAppPath != "" {
return claudeDesktopOpenAppPath(restartAppPath)
}
return claudeDesktopOpenApp()
}
func waitForClaudeDesktopExit(timeout time.Duration) error {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
if !claudeDesktopIsRunning() {
return nil
}
claudeDesktopSleep(200 * time.Millisecond)
}
return fmt.Errorf("Claude Desktop did not quit; quit it manually and re-run the command")
}
func defaultClaudeDesktopIsRunning() bool {
switch claudeDesktopGOOS {
case "darwin":
out, err := exec.Command("pgrep", "-f", "Claude.app/Contents/MacOS/Claude").Output()
return err == nil && strings.TrimSpace(string(out)) != ""
case "windows":
out, err := exec.Command("powershell.exe", "-NoProfile", "-Command", `(Get-Process claude -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 } | Select-Object -First 1).Id`).Output()
return err == nil && strings.TrimSpace(string(out)) != ""
default:
return false
}
}
func defaultClaudeDesktopOpenApp() error {
switch claudeDesktopGOOS {
case "windows":
if path := claudeDesktopAppPath(); path != "" {
return claudeDesktopOpenAppPath(path)
}
if path := claudeDesktopRunningAppPath(); path != "" {
return claudeDesktopOpenAppPath(path)
}
return fmt.Errorf("Claude Desktop executable was not found; open Claude Desktop manually once and re-run 'ollama launch claude-desktop'")
case "darwin":
return openClaudeDesktopDarwin()
default:
return claudeDesktopSupported()
}
}
func defaultClaudeDesktopOpenAppPath(path string) error {
switch claudeDesktopGOOS {
case "windows":
return exec.Command("powershell.exe", "-NoProfile", "-Command", "Start-Process -FilePath "+quotePowerShellString(path)).Run()
case "darwin":
return openClaudeDesktopDarwin()
default:
return claudeDesktopSupported()
}
}
func openClaudeDesktopDarwin() error {
cmd := exec.Command("open", "-a", "Claude")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
func defaultClaudeDesktopRunningAppPath() string {
if claudeDesktopGOOS != "windows" {
return ""
}
script := `(Get-Process claude -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 -and $_.Path } | Select-Object -First 1 -ExpandProperty Path)`
out, err := exec.Command("powershell.exe", "-NoProfile", "-Command", script).Output()
if err != nil {
return ""
}
return strings.TrimSpace(string(out))
}
func defaultClaudeDesktopQuitApp() error {
if claudeDesktopGOOS == "windows" {
script := `Get-Process claude -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 } | ForEach-Object { [void]$_.CloseMainWindow() }`
return exec.Command("powershell.exe", "-NoProfile", "-Command", script).Run()
}
return exec.Command("osascript", "-e", `tell application "Claude" to quit`).Run()
}
func quotePowerShellString(s string) string {
return "'" + strings.ReplaceAll(s, "'", "''") + "'"
}
+938
View File
@@ -0,0 +1,938 @@
package launch
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func withClaudeDesktopPlatform(t *testing.T, goos string) {
t.Helper()
old := claudeDesktopGOOS
claudeDesktopGOOS = goos
t.Cleanup(func() {
claudeDesktopGOOS = old
})
}
func withClaudeDesktopValidation(t *testing.T, fn func(context.Context, string) error) {
t.Helper()
old := claudeDesktopValidateAPIKey
claudeDesktopValidateAPIKey = fn
t.Cleanup(func() {
claudeDesktopValidateAPIKey = old
})
}
func withClaudeDesktopPrompt(t *testing.T, fn func() (string, error)) {
t.Helper()
old := claudeDesktopPromptAPIKey
claudeDesktopPromptAPIKey = fn
t.Cleanup(func() {
claudeDesktopPromptAPIKey = old
})
}
func withClaudeDesktopProcessHooks(t *testing.T, running func() bool, quit func() error, open func() error) {
t.Helper()
oldRunning := claudeDesktopIsRunning
oldQuit := claudeDesktopQuitApp
oldOpen := claudeDesktopOpenApp
oldOpenPath := claudeDesktopOpenAppPath
oldRunningPath := claudeDesktopRunningAppPath
oldSleep := claudeDesktopSleep
claudeDesktopIsRunning = running
claudeDesktopQuitApp = quit
claudeDesktopOpenApp = open
claudeDesktopOpenAppPath = oldOpenPath
claudeDesktopRunningAppPath = oldRunningPath
claudeDesktopSleep = func(time.Duration) {}
t.Cleanup(func() {
claudeDesktopIsRunning = oldRunning
claudeDesktopQuitApp = oldQuit
claudeDesktopOpenApp = oldOpen
claudeDesktopOpenAppPath = oldOpenPath
claudeDesktopRunningAppPath = oldRunningPath
claudeDesktopSleep = oldSleep
})
}
func claudeDesktopReadJSON(t *testing.T, path string) map[string]any {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
var cfg map[string]any
if err := json.Unmarshal(data, &cfg); err != nil {
t.Fatalf("parse %s: %v", path, err)
}
return cfg
}
func TestClaudeDesktopIntegration(t *testing.T) {
c := &ClaudeDesktop{}
t.Run("implements Runner", func(t *testing.T) {
var _ Runner = c
})
t.Run("implements managed autodiscovery integration", func(t *testing.T) {
var _ ManagedAutodiscoveryIntegration = c
})
t.Run("does not use local Ollama Cloud auth gate", func(t *testing.T) {
if _, ok := any(c).(ManagedAutodiscoveryCloudIntegration); ok {
t.Fatal("Claude Desktop should validate OLLAMA_API_KEY directly instead of requiring local Ollama Cloud sign-in")
}
})
t.Run("implements restore", func(t *testing.T) {
var _ RestorableIntegration = c
})
t.Run("has restore hint", func(t *testing.T) {
var _ RestoreHintIntegration = c
if !strings.Contains(c.RestoreHint(), "--restore") {
t.Fatalf("expected restore hint to mention --restore, got %q", c.RestoreHint())
}
if strings.Contains(c.RestoreHint(), "Tip:") {
t.Fatalf("restore hint should not use Tip wording, got %q", c.RestoreHint())
}
})
t.Run("has success messages", func(t *testing.T) {
var _ ConfigurationSuccessIntegration = c
var _ RestoreSuccessIntegration = c
if got := c.ConfigurationSuccessMessage(); got != "Claude Desktop profile changed to Ollama Cloud.\nTo restore the usual Claude profile, run: ollama launch claude-desktop --restore" {
t.Fatalf("configuration success message = %q", got)
}
if got := c.RestoreSuccessMessage(); got != "Claude Desktop restored to the usual Claude profile." {
t.Fatalf("restore success message = %q", got)
}
})
t.Run("skips local model readiness", func(t *testing.T) {
var _ ManagedModelReadinessSkipper = c
if !c.SkipModelReadiness() {
t.Fatal("expected Claude Desktop to skip local model readiness")
}
})
}
func TestLaunchIntegration_ClaudeDesktopDoesNotRequireLocalCloudSignIn(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
withInteractiveSession(t, true)
withLauncherHooks(t)
t.Setenv("OLLAMA_API_KEY", "test-api-key")
if err := os.MkdirAll(filepath.Join(tmpDir, "Applications", "Claude.app"), 0o755); err != nil {
t.Fatal(err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/status":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"not found"}`)
case "/api/me":
w.WriteHeader(http.StatusUnauthorized)
fmt.Fprint(w, `{"error":"unauthorized","signin_url":"https://example.com/signin"}`)
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
var validatedKey string
withClaudeDesktopValidation(t, func(_ context.Context, key string) error {
validatedKey = key
return nil
})
DefaultSignIn = func(modelName, signInURL string) (string, error) {
t.Fatalf("Claude Desktop launch should not require local Ollama Cloud sign-in, got %s at %s", modelName, signInURL)
return "", nil
}
var openCalls int
withClaudeDesktopProcessHooks(t,
func() bool { return false },
func() error { return nil },
func() error {
openCalls++
return nil
},
)
if err := LaunchIntegration(context.Background(), IntegrationLaunchRequest{Name: "claude-desktop"}); err != nil {
t.Fatalf("LaunchIntegration returned error: %v", err)
}
if validatedKey != "test-api-key" {
t.Fatalf("validated key = %q, want test API key", validatedKey)
}
if openCalls != 1 {
t.Fatalf("open calls = %d, want 1", openCalls)
}
}
func TestClaudeDesktopConfigureWritesOllamaCloudProfile(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "test-api-key")
var validatedKey string
withClaudeDesktopValidation(t, func(_ context.Context, key string) error {
validatedKey = key
return nil
})
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.desktopConfig), 0o755); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.meta), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.desktopConfig, []byte(`{"existing":true}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.meta, []byte(`{"entries":[{"id":"custom","name":"Custom"}]}`), 0o644); err != nil {
t.Fatal(err)
}
if err := (&ClaudeDesktop{}).ConfigureAutodiscovery(); err != nil {
t.Fatalf("Configure returned error: %v", err)
}
if validatedKey != "test-api-key" {
t.Fatalf("validated key = %q, want test API key", validatedKey)
}
desktopConfig := claudeDesktopReadJSON(t, paths.desktopConfig)
if desktopConfig["existing"] != true {
t.Fatalf("existing desktop config key was not preserved: %v", desktopConfig)
}
if desktopConfig["deploymentMode"] != "3p" {
t.Fatalf("deploymentMode = %v, want 3p", desktopConfig["deploymentMode"])
}
normalConfig := claudeDesktopReadJSON(t, paths.normalConfig)
if normalConfig["deploymentMode"] != "3p" {
t.Fatalf("normal deploymentMode = %v, want 3p", normalConfig["deploymentMode"])
}
meta := claudeDesktopReadJSON(t, paths.meta)
if meta["appliedId"] != claudeDesktopProfileID {
t.Fatalf("appliedId = %v, want %s", meta["appliedId"], claudeDesktopProfileID)
}
entries, _ := meta["entries"].([]any)
if len(entries) != 2 {
t.Fatalf("entries len = %d, want 2: %v", len(entries), entries)
}
profile := claudeDesktopReadJSON(t, paths.profile)
if profile["inferenceProvider"] != "gateway" {
t.Fatalf("inferenceProvider = %v, want gateway", profile["inferenceProvider"])
}
if profile["inferenceGatewayBaseUrl"] != claudeDesktopGatewayBaseURL {
t.Fatalf("base URL = %v, want %s", profile["inferenceGatewayBaseUrl"], claudeDesktopGatewayBaseURL)
}
if profile["inferenceGatewayApiKey"] != "test-api-key" {
t.Fatal("expected configured API key to be written")
}
if profile["inferenceGatewayAuthScheme"] != "bearer" {
t.Fatalf("auth scheme = %v, want bearer", profile["inferenceGatewayAuthScheme"])
}
if profile["disableDeploymentModeChooser"] != true {
t.Fatalf("disableDeploymentModeChooser = %v, want true", profile["disableDeploymentModeChooser"])
}
if _, ok := profile["inferenceModels"]; ok {
t.Fatalf("inferenceModels should be omitted so Claude can discover models, got %v", profile["inferenceModels"])
}
}
func TestClaudeDesktopConfigureAutodiscoveryRemovesExistingModelCatalog(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "test-api-key")
withClaudeDesktopValidation(t, func(context.Context, string) error { return nil })
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.profile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.profile, []byte(`{"inferenceModels":["qwen3.5"],"inferenceGatewayApiKey":"old"}`), 0o644); err != nil {
t.Fatal(err)
}
if err := (&ClaudeDesktop{}).ConfigureAutodiscovery(); err != nil {
t.Fatalf("ConfigureAutodiscovery returned error: %v", err)
}
profile := claudeDesktopReadJSON(t, paths.profile)
if _, ok := profile["inferenceModels"]; ok {
t.Fatalf("inferenceModels should be removed, got %v", profile["inferenceModels"])
}
if profile["inferenceGatewayApiKey"] != "test-api-key" {
t.Fatal("expected env API key to replace the old key")
}
}
func TestClaudeDesktopWindowsConfigPathsUseLocalAppData(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "LocalAppData"))
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if want := filepath.Join(tmpDir, "LocalAppData", "Claude-3p", "claude_desktop_config.json"); paths.desktopConfig != want {
t.Fatalf("desktop config = %q, want %q", paths.desktopConfig, want)
}
if want := filepath.Join(tmpDir, "LocalAppData", "Claude", "claude_desktop_config.json"); paths.normalConfig != want {
t.Fatalf("normal config = %q, want %q", paths.normalConfig, want)
}
}
func TestClaudeDesktopWindowsConfigPathsFallbackToNestProfile(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
if err := os.MkdirAll(filepath.Join(local, "Claude Nest-3p"), 0o755); err != nil {
t.Fatal(err)
}
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if want := filepath.Join(local, "Claude Nest-3p", "claude_desktop_config.json"); paths.desktopConfig != want {
t.Fatalf("desktop config = %q, want %q", paths.desktopConfig, want)
}
}
func TestClaudeDesktopAutodiscoveryConfiguredOnWindows(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "LocalAppData"))
t.Setenv("OLLAMA_API_KEY", "test-api-key")
withClaudeDesktopValidation(t, func(context.Context, string) error { return nil })
c := &ClaudeDesktop{}
if err := c.ConfigureAutodiscovery(); err != nil {
t.Fatalf("Configure returned error: %v", err)
}
if !c.AutodiscoveryConfigured() {
t.Fatal("expected Claude Desktop autodiscovery config to be detected on Windows")
}
}
func TestClaudeDesktopConfigureAutodiscoveryTouchesAllWindowsProfileCandidates(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
t.Setenv("OLLAMA_API_KEY", "test-api-key")
withClaudeDesktopValidation(t, func(context.Context, string) error { return nil })
targets, err := claudeDesktopTargetPaths()
if err != nil {
t.Fatal(err)
}
if len(targets.normalConfigs) != 2 {
t.Fatalf("normal config target count = %d, want 2", len(targets.normalConfigs))
}
if len(targets.thirdPartyProfiles) != 2 {
t.Fatalf("third-party target count = %d, want 2", len(targets.thirdPartyProfiles))
}
c := &ClaudeDesktop{}
if err := c.ConfigureAutodiscovery(); err != nil {
t.Fatalf("ConfigureAutodiscovery returned error: %v", err)
}
for _, path := range targets.normalConfigs {
cfg := claudeDesktopReadJSON(t, path)
if cfg["deploymentMode"] != "3p" {
t.Fatalf("%s deploymentMode = %v, want 3p", path, cfg["deploymentMode"])
}
}
for _, target := range targets.thirdPartyProfiles {
cfg := claudeDesktopReadJSON(t, target.desktopConfig)
if cfg["deploymentMode"] != "3p" {
t.Fatalf("%s deploymentMode = %v, want 3p", target.desktopConfig, cfg["deploymentMode"])
}
meta := claudeDesktopReadJSON(t, target.meta)
if meta["appliedId"] != claudeDesktopProfileID {
t.Fatalf("%s appliedId = %v, want %s", target.meta, meta["appliedId"], claudeDesktopProfileID)
}
profile := claudeDesktopReadJSON(t, target.profile)
if profile["inferenceProvider"] != "gateway" {
t.Fatalf("%s inferenceProvider = %v, want gateway", target.profile, profile["inferenceProvider"])
}
if profile["inferenceGatewayBaseUrl"] != claudeDesktopGatewayBaseURL {
t.Fatalf("%s base URL = %v, want %s", target.profile, profile["inferenceGatewayBaseUrl"], claudeDesktopGatewayBaseURL)
}
if profile["inferenceGatewayApiKey"] != "test-api-key" {
t.Fatalf("%s should contain the configured API key", target.profile)
}
if _, ok := profile["inferenceModels"]; ok {
t.Fatalf("%s inferenceModels should be omitted, got %v", target.profile, profile["inferenceModels"])
}
}
if !c.AutodiscoveryConfigured() {
t.Fatal("expected all Windows profile candidates to be considered configured")
}
if err := writeClaudeDesktopDeploymentMode(targets.thirdPartyProfiles[1].desktopConfig, "1p"); err != nil {
t.Fatal(err)
}
if c.AutodiscoveryConfigured() {
t.Fatal("expected a stale Windows candidate to force reconfiguration")
}
}
func TestClaudeDesktopInstalledOnWindowsRecognizesLocalProfileDir(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
withClaudeDesktopProcessHooks(t, func() bool { return false }, func() error { return nil }, func() error { return nil })
if err := os.MkdirAll(filepath.Join(local, "Claude-3p"), 0o755); err != nil {
t.Fatal(err)
}
if !claudeDesktopInstalled() {
t.Fatal("expected Claude Desktop to be installed when the Windows profile directory exists")
}
}
func TestClaudeDesktopWindowsAppPathFindsAnthropicClaudeInstall(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
want := filepath.Join(local, "AnthropicClaude", "app-1.2.3", "Claude.exe")
if err := os.MkdirAll(filepath.Dir(want), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(want, []byte(""), 0o755); err != nil {
t.Fatal(err)
}
if got := claudeDesktopAppPath(); got != want {
t.Fatalf("claudeDesktopAppPath() = %q, want %q", got, want)
}
}
func TestWaitForClaudeDesktopExitUsesRunningHook(t *testing.T) {
withClaudeDesktopPlatform(t, "windows")
runningChecks := 0
withClaudeDesktopProcessHooks(t,
func() bool {
runningChecks++
return runningChecks == 1
},
func() error { return nil },
func() error { return nil },
)
if err := waitForClaudeDesktopExit(time.Second); err != nil {
t.Fatalf("waitForClaudeDesktopExit returned error: %v", err)
}
if runningChecks < 2 {
t.Fatalf("expected running hook to be checked until the visible window exits, got %d checks", runningChecks)
}
}
func TestClaudeDesktopWindowsRestartUsesCapturedDesktopPath(t *testing.T) {
withClaudeDesktopPlatform(t, "windows")
restoreConfirm := withLaunchConfirmPolicy(launchConfirmPolicy{yes: true})
defer restoreConfirm()
desktopPath := `C:\Users\parth\AppData\Local\AnthropicClaude\app-1.2.3\Claude.exe`
running := true
var openedPath string
withClaudeDesktopProcessHooks(t,
func() bool { return running },
func() error {
running = false
return nil
},
func() error {
t.Fatal("expected restart to open the captured Desktop executable path, not the generic launcher")
return nil
},
)
claudeDesktopRunningAppPath = func() string { return desktopPath }
claudeDesktopOpenAppPath = func(path string) error {
openedPath = path
return nil
}
if err := (&ClaudeDesktop{}).Run("qwen3.5", nil); err != nil {
t.Fatalf("Run returned error: %v", err)
}
if openedPath != desktopPath {
t.Fatalf("opened path = %q, want %q", openedPath, desktopPath)
}
}
func TestClaudeDesktopWindowsOpenDoesNotFallBackToClaudeCommand(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
t.Setenv("LOCALAPPDATA", filepath.Join(tmpDir, "LocalAppData"))
oldRunningPath := claudeDesktopRunningAppPath
claudeDesktopRunningAppPath = func() string { return "" }
t.Cleanup(func() { claudeDesktopRunningAppPath = oldRunningPath })
err := defaultClaudeDesktopOpenApp()
if err == nil || !strings.Contains(err.Error(), "Claude Desktop executable was not found") {
t.Fatalf("defaultClaudeDesktopOpenApp error = %v, want executable-not-found error", err)
}
}
func TestClaudeDesktopConfigureStopsBeforeWriteWhenKeyValidationFails(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "bad-key")
withClaudeDesktopValidation(t, func(context.Context, string) error {
return errors.New("invalid key")
})
err := (&ClaudeDesktop{}).ConfigureAutodiscovery()
if err == nil || !strings.Contains(err.Error(), "invalid key") {
t.Fatalf("Configure error = %v, want invalid key", err)
}
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if _, err := os.Stat(paths.desktopConfig); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("desktop config should not be written after validation failure, stat err = %v", err)
}
}
func TestValidateClaudeDesktopAPIKeyUsesClaudeModelsRoute(t *testing.T) {
oldClient := claudeDesktopHTTPClient
var gotPath, gotAuth string
claudeDesktopHTTPClient = &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
gotPath = req.URL.Path
gotAuth = req.Header.Get("Authorization")
return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader(`{"data":[]}`)),
Header: make(http.Header),
}, nil
})}
t.Cleanup(func() {
claudeDesktopHTTPClient = oldClient
})
if err := validateClaudeDesktopAPIKey(context.Background(), "test-key"); err != nil {
t.Fatalf("validateClaudeDesktopAPIKey returned error: %v", err)
}
if gotPath != "/v1/models" {
t.Fatalf("validation path = %q, want /v1/models", gotPath)
}
if gotAuth != "Bearer test-key" {
t.Fatalf("Authorization header = %q, want bearer key", gotAuth)
}
}
func TestValidateClaudeDesktopAPIKeyHidesInvalidHeaderDetails(t *testing.T) {
err := validateClaudeDesktopAPIKey(context.Background(), "bad\nkey")
if err == nil {
t.Fatal("expected validation error for key with newline")
}
if !strings.Contains(err.Error(), "could not verify Ollama API key") {
t.Fatalf("validation error = %v, want friendly verification message", err)
}
if strings.Contains(err.Error(), "invalid header") || strings.Contains(err.Error(), "net/http") {
t.Fatalf("validation error should not expose transport internals: %v", err)
}
if !strings.Contains(err.Error(), "https://ollama.com/settings/keys") {
t.Fatalf("validation error should include settings link: %v", err)
}
}
func TestClaudeDesktopConfigureRequiresAPIKey(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "")
withClaudeDesktopValidation(t, func(context.Context, string) error {
t.Fatal("validation should not run without an API key")
return nil
})
err := (&ClaudeDesktop{}).ConfigureAutodiscovery()
if err == nil || !strings.Contains(err.Error(), "OLLAMA_API_KEY is required") {
t.Fatalf("Configure error = %v, want missing key guidance", err)
}
}
func TestClaudeDesktopAPIKeyPromptIncludesSettingsLink(t *testing.T) {
prompt := claudeDesktopAPIKeyPrompt()
if !strings.Contains(prompt, "Enter Ollama API key") {
t.Fatalf("prompt should ask for the API key, got %q", prompt)
}
if !strings.Contains(prompt, "https://ollama.com/settings/keys") {
t.Fatalf("prompt should include API key settings link, got %q", prompt)
}
}
func TestClaudeDesktopConfigureReusesExistingAPIKey(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "")
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.profile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.profile, []byte(`{"inferenceGatewayApiKey":"existing-key"}`), 0o644); err != nil {
t.Fatal(err)
}
var validatedKey string
withClaudeDesktopValidation(t, func(_ context.Context, key string) error {
validatedKey = key
return nil
})
if err := (&ClaudeDesktop{}).ConfigureAutodiscovery(); err != nil {
t.Fatalf("ConfigureAutodiscovery returned error: %v", err)
}
if validatedKey != "existing-key" {
t.Fatalf("validated key = %q, want existing-key", validatedKey)
}
}
func TestClaudeDesktopConfigureReplacesInvalidExistingAPIKey(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
withInteractiveSession(t, true)
t.Setenv("OLLAMA_API_KEY", "")
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.profile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.profile, []byte(`{"inferenceGatewayApiKey":"stale-key"}`), 0o644); err != nil {
t.Fatal(err)
}
var validated []string
withClaudeDesktopValidation(t, func(_ context.Context, key string) error {
validated = append(validated, key)
if key == "stale-key" {
return errors.New("invalid key")
}
return nil
})
withClaudeDesktopPrompt(t, func() (string, error) {
return "replacement-key", nil
})
if err := (&ClaudeDesktop{}).ConfigureAutodiscovery(); err != nil {
t.Fatalf("ConfigureAutodiscovery returned error: %v", err)
}
if diff := compareStrings(validated, []string{"stale-key", "replacement-key"}); diff != "" {
t.Fatalf("validated keys mismatch: %s", diff)
}
profile := claudeDesktopReadJSON(t, paths.profile)
if profile["inferenceGatewayApiKey"] != "replacement-key" {
t.Fatalf("configured key = %v, want replacement-key", profile["inferenceGatewayApiKey"])
}
}
func TestClaudeDesktopConfigureReusesExistingAPIKeyFromAnyWindowsProfile(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
t.Setenv("OLLAMA_API_KEY", "")
targets, err := claudeDesktopTargetPaths()
if err != nil {
t.Fatal(err)
}
fallbackProfile := targets.thirdPartyProfiles[1].profile
if err := os.MkdirAll(filepath.Dir(fallbackProfile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(fallbackProfile, []byte(`{"inferenceGatewayApiKey":"fallback-key"}`), 0o644); err != nil {
t.Fatal(err)
}
var validatedKey string
withClaudeDesktopValidation(t, func(_ context.Context, key string) error {
validatedKey = key
return nil
})
if err := (&ClaudeDesktop{}).ConfigureAutodiscovery(); err != nil {
t.Fatalf("ConfigureAutodiscovery returned error: %v", err)
}
if validatedKey != "fallback-key" {
t.Fatalf("validated key = %q, want fallback-key", validatedKey)
}
for _, target := range targets.thirdPartyProfiles {
profile := claudeDesktopReadJSON(t, target.profile)
if profile["inferenceGatewayApiKey"] != "fallback-key" {
t.Fatalf("%s should reuse fallback key, got %v", target.profile, profile["inferenceGatewayApiKey"])
}
}
}
func TestClaudeDesktopAutodiscoveryConfiguredRequiresAppliedOllamaProfile(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "test-api-key")
withClaudeDesktopValidation(t, func(context.Context, string) error { return nil })
c := &ClaudeDesktop{}
if err := c.ConfigureAutodiscovery(); err != nil {
t.Fatalf("Configure returned error: %v", err)
}
if !c.AutodiscoveryConfigured() {
t.Fatal("expected Claude Desktop autodiscovery config to be detected")
}
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.meta, []byte(`{"appliedId":"custom"}`), 0o644); err != nil {
t.Fatal(err)
}
if c.AutodiscoveryConfigured() {
t.Fatal("expected another applied profile to hide Claude Desktop autodiscovery config")
}
}
func TestClaudeDesktopAutodiscoveryConfiguredRequiresAPIKey(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
t.Setenv("OLLAMA_API_KEY", "test-api-key")
withClaudeDesktopValidation(t, func(context.Context, string) error { return nil })
c := &ClaudeDesktop{}
if err := c.ConfigureAutodiscovery(); err != nil {
t.Fatalf("Configure returned error: %v", err)
}
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
profile := claudeDesktopReadJSON(t, paths.profile)
delete(profile, "inferenceGatewayApiKey")
data, err := json.Marshal(profile)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.profile, data, 0o644); err != nil {
t.Fatal(err)
}
if c.AutodiscoveryConfigured() {
t.Fatal("expected missing gateway API key to force Claude Desktop reconfiguration")
}
}
func TestClaudeDesktopRestoreSwitchesBackToFirstPartyMode(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "darwin")
withClaudeDesktopProcessHooks(t, func() bool { return false }, func() error { return nil }, func() error { return nil })
paths, err := claudeDesktopConfigPaths()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(paths.profile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.meta, []byte(`{"appliedId":"`+claudeDesktopProfileID+`","entries":[{"id":"`+claudeDesktopProfileID+`","name":"Ollama"}]}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(paths.profile, []byte(`{"disableDeploymentModeChooser":true,"inferenceGatewayApiKey":"keep","inferenceProvider":"gateway","inferenceGatewayBaseUrl":"https://ollama.com","inferenceGatewayAuthScheme":"bearer","inferenceModels":["legacy"]}`), 0o644); err != nil {
t.Fatal(err)
}
if err := (&ClaudeDesktop{}).Restore(); err != nil {
t.Fatalf("Restore returned error: %v", err)
}
desktopConfig := claudeDesktopReadJSON(t, paths.desktopConfig)
if desktopConfig["deploymentMode"] != "1p" {
t.Fatalf("deploymentMode = %v, want 1p", desktopConfig["deploymentMode"])
}
normalConfig := claudeDesktopReadJSON(t, paths.normalConfig)
if normalConfig["deploymentMode"] != "1p" {
t.Fatalf("normal deploymentMode = %v, want 1p", normalConfig["deploymentMode"])
}
profile := claudeDesktopReadJSON(t, paths.profile)
if profile["disableDeploymentModeChooser"] != false {
t.Fatalf("disableDeploymentModeChooser = %v, want false", profile["disableDeploymentModeChooser"])
}
if profile["inferenceGatewayApiKey"] != "keep" {
t.Fatal("restore should leave existing Ollama profile credentials in place")
}
for _, key := range []string{"inferenceProvider", "inferenceGatewayBaseUrl", "inferenceGatewayAuthScheme", "inferenceModels"} {
if _, ok := profile[key]; ok {
t.Fatalf("restore should clear stale %s from the Ollama profile: %v", key, profile)
}
}
meta := claudeDesktopReadJSON(t, paths.meta)
if _, ok := meta["appliedId"]; ok {
t.Fatalf("restore should clear the applied Ollama third-party profile: %v", meta)
}
if (&ClaudeDesktop{}).AutodiscoveryConfigured() {
t.Fatal("restore should leave Claude Desktop autodiscovery unconfigured")
}
}
func TestClaudeDesktopRestoreTouchesAllWindowsProfileCandidates(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
withClaudeDesktopPlatform(t, "windows")
local := filepath.Join(tmpDir, "LocalAppData")
t.Setenv("LOCALAPPDATA", local)
withClaudeDesktopProcessHooks(t, func() bool { return false }, func() error { return nil }, func() error { return nil })
targets, err := claudeDesktopTargetPaths()
if err != nil {
t.Fatal(err)
}
if len(targets.normalConfigs) != 2 {
t.Fatalf("normal config target count = %d, want 2", len(targets.normalConfigs))
}
if len(targets.thirdPartyProfiles) != 2 {
t.Fatalf("third-party target count = %d, want 2", len(targets.thirdPartyProfiles))
}
for _, target := range targets.thirdPartyProfiles {
if err := os.MkdirAll(filepath.Dir(target.profile), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(target.meta, []byte(`{"appliedId":"`+claudeDesktopProfileID+`","entries":[{"id":"`+claudeDesktopProfileID+`","name":"Ollama"}]}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(target.profile, []byte(`{"disableDeploymentModeChooser":true,"inferenceGatewayApiKey":"keep","inferenceProvider":"gateway","inferenceGatewayBaseUrl":"https://ollama.com","inferenceGatewayAuthScheme":"bearer","inferenceModels":["legacy"]}`), 0o644); err != nil {
t.Fatal(err)
}
}
if err := (&ClaudeDesktop{}).Restore(); err != nil {
t.Fatalf("Restore returned error: %v", err)
}
for _, path := range targets.normalConfigs {
cfg := claudeDesktopReadJSON(t, path)
if cfg["deploymentMode"] != "1p" {
t.Fatalf("%s deploymentMode = %v, want 1p", path, cfg["deploymentMode"])
}
}
for _, target := range targets.thirdPartyProfiles {
cfg := claudeDesktopReadJSON(t, target.desktopConfig)
if cfg["deploymentMode"] != "1p" {
t.Fatalf("%s deploymentMode = %v, want 1p", target.desktopConfig, cfg["deploymentMode"])
}
meta := claudeDesktopReadJSON(t, target.meta)
if _, ok := meta["appliedId"]; ok {
t.Fatalf("%s should not keep the Ollama applied profile: %v", target.meta, meta)
}
profile := claudeDesktopReadJSON(t, target.profile)
if profile["disableDeploymentModeChooser"] != false {
t.Fatalf("%s disableDeploymentModeChooser = %v, want false", target.profile, profile["disableDeploymentModeChooser"])
}
if profile["inferenceGatewayApiKey"] != "keep" {
t.Fatalf("%s should preserve gateway API key", target.profile)
}
for _, key := range []string{"inferenceProvider", "inferenceGatewayBaseUrl", "inferenceGatewayAuthScheme", "inferenceModels"} {
if _, ok := profile[key]; ok {
t.Fatalf("%s should clear stale %s: %v", target.profile, key, profile)
}
}
}
}
func TestClaudeDesktopRunRestartsRunningAppWhenConfirmed(t *testing.T) {
withClaudeDesktopPlatform(t, "darwin")
restoreConfirm := withLaunchConfirmPolicy(launchConfirmPolicy{yes: true})
defer restoreConfirm()
running := true
var quitCalls, openCalls int
withClaudeDesktopProcessHooks(t,
func() bool { return running },
func() error {
quitCalls++
running = false
return nil
},
func() error {
openCalls++
return nil
},
)
if err := (&ClaudeDesktop{}).Run("qwen3.5", nil); err != nil {
t.Fatalf("Run returned error: %v", err)
}
if quitCalls != 1 || openCalls != 1 {
t.Fatalf("quit/open calls = %d/%d, want 1/1", quitCalls, openCalls)
}
}
func TestClaudeDesktopRunRejectsExtraArgs(t *testing.T) {
withClaudeDesktopPlatform(t, "darwin")
err := (&ClaudeDesktop{}).Run("qwen3.5", []string{"--foo"})
if err == nil || !strings.Contains(err.Error(), "does not accept extra arguments") {
t.Fatalf("Run error = %v, want extra args rejection", err)
}
}
@@ -1,4 +1,4 @@
package config
package launch
import (
"os"
@@ -117,10 +117,7 @@ func TestClaudeModelEnvVars(t *testing.T) {
return m
}
t.Run("falls back to model param when no aliases saved", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Run("maps all Claude model env vars to the provided model", func(t *testing.T) {
got := envMap(c.modelEnvVars("llama3.2"))
if got["ANTHROPIC_DEFAULT_OPUS_MODEL"] != "llama3.2" {
t.Errorf("OPUS = %q, want llama3.2", got["ANTHROPIC_DEFAULT_OPUS_MODEL"])
@@ -134,65 +131,41 @@ func TestClaudeModelEnvVars(t *testing.T) {
if got["CLAUDE_CODE_SUBAGENT_MODEL"] != "llama3.2" {
t.Errorf("SUBAGENT = %q, want llama3.2", got["CLAUDE_CODE_SUBAGENT_MODEL"])
}
})
t.Run("uses primary alias for opus sonnet and subagent", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
SaveIntegration("claude", []string{"qwen3:8b"})
saveAliases("claude", map[string]string{"primary": "qwen3:8b"})
got := envMap(c.modelEnvVars("qwen3:8b"))
if got["ANTHROPIC_DEFAULT_OPUS_MODEL"] != "qwen3:8b" {
t.Errorf("OPUS = %q, want qwen3:8b", got["ANTHROPIC_DEFAULT_OPUS_MODEL"])
}
if got["ANTHROPIC_DEFAULT_SONNET_MODEL"] != "qwen3:8b" {
t.Errorf("SONNET = %q, want qwen3:8b", got["ANTHROPIC_DEFAULT_SONNET_MODEL"])
}
if got["ANTHROPIC_DEFAULT_HAIKU_MODEL"] != "qwen3:8b" {
t.Errorf("HAIKU = %q, want qwen3:8b (no fast alias)", got["ANTHROPIC_DEFAULT_HAIKU_MODEL"])
}
if got["CLAUDE_CODE_SUBAGENT_MODEL"] != "qwen3:8b" {
t.Errorf("SUBAGENT = %q, want qwen3:8b", got["CLAUDE_CODE_SUBAGENT_MODEL"])
if got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] != "" {
t.Errorf("AUTO_COMPACT_WINDOW = %q, want empty for local models", got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"])
}
})
t.Run("uses fast alias for haiku", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
SaveIntegration("claude", []string{"llama3.2:70b"})
saveAliases("claude", map[string]string{
"primary": "llama3.2:70b",
"fast": "llama3.2:8b",
})
got := envMap(c.modelEnvVars("llama3.2:70b"))
if got["ANTHROPIC_DEFAULT_OPUS_MODEL"] != "llama3.2:70b" {
t.Errorf("OPUS = %q, want llama3.2:70b", got["ANTHROPIC_DEFAULT_OPUS_MODEL"])
t.Run("supports empty model", func(t *testing.T) {
got := envMap(c.modelEnvVars(""))
if got["ANTHROPIC_DEFAULT_OPUS_MODEL"] != "" {
t.Errorf("OPUS = %q, want empty", got["ANTHROPIC_DEFAULT_OPUS_MODEL"])
}
if got["ANTHROPIC_DEFAULT_SONNET_MODEL"] != "llama3.2:70b" {
t.Errorf("SONNET = %q, want llama3.2:70b", got["ANTHROPIC_DEFAULT_SONNET_MODEL"])
if got["ANTHROPIC_DEFAULT_SONNET_MODEL"] != "" {
t.Errorf("SONNET = %q, want empty", got["ANTHROPIC_DEFAULT_SONNET_MODEL"])
}
if got["ANTHROPIC_DEFAULT_HAIKU_MODEL"] != "llama3.2:8b" {
t.Errorf("HAIKU = %q, want llama3.2:8b", got["ANTHROPIC_DEFAULT_HAIKU_MODEL"])
if got["ANTHROPIC_DEFAULT_HAIKU_MODEL"] != "" {
t.Errorf("HAIKU = %q, want empty", got["ANTHROPIC_DEFAULT_HAIKU_MODEL"])
}
if got["CLAUDE_CODE_SUBAGENT_MODEL"] != "llama3.2:70b" {
t.Errorf("SUBAGENT = %q, want llama3.2:70b", got["CLAUDE_CODE_SUBAGENT_MODEL"])
if got["CLAUDE_CODE_SUBAGENT_MODEL"] != "" {
t.Errorf("SUBAGENT = %q, want empty", got["CLAUDE_CODE_SUBAGENT_MODEL"])
}
if got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] != "" {
t.Errorf("AUTO_COMPACT_WINDOW = %q, want empty", got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"])
}
})
t.Run("alias primary overrides model param", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
t.Run("sets auto compact window for known cloud models", func(t *testing.T) {
got := envMap(c.modelEnvVars("glm-5:cloud"))
if got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] != "202752" {
t.Errorf("AUTO_COMPACT_WINDOW = %q, want 202752", got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"])
}
})
SaveIntegration("claude", []string{"saved-model"})
saveAliases("claude", map[string]string{"primary": "saved-model"})
got := envMap(c.modelEnvVars("different-model"))
if got["ANTHROPIC_DEFAULT_OPUS_MODEL"] != "saved-model" {
t.Errorf("OPUS = %q, want saved-model", got["ANTHROPIC_DEFAULT_OPUS_MODEL"])
t.Run("does not set auto compact window for unknown cloud models", func(t *testing.T) {
got := envMap(c.modelEnvVars("unknown-model:cloud"))
if got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] != "" {
t.Errorf("AUTO_COMPACT_WINDOW = %q, want empty", got["CLAUDE_CODE_AUTO_COMPACT_WINDOW"])
}
})
}
+4 -23
View File
@@ -1,14 +1,13 @@
package config
package launch
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"github.com/ollama/ollama/cmd/internal/fileutil"
"github.com/ollama/ollama/envconfig"
)
@@ -22,24 +21,6 @@ func (c *Cline) Run(model string, args []string) error {
return fmt.Errorf("cline is not installed, install with: npm install -g cline")
}
models := []string{model}
if config, err := loadIntegration("cline"); err == nil && len(config.Models) > 0 {
models = config.Models
}
var err error
models, err = resolveEditorModels("cline", models, func() ([]string, error) {
return selectModels(context.Background(), "cline", "")
})
if errors.Is(err, errCancelled) {
return nil
}
if err != nil {
return err
}
if err := c.Edit(models); err != nil {
return fmt.Errorf("setup failed: %w", err)
}
cmd := exec.Command("cline", args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
@@ -97,7 +78,7 @@ func (c *Cline) Edit(models []string) error {
if err != nil {
return err
}
return writeWithBackup(configPath, data)
return fileutil.WriteWithBackup(configPath, data, "cline")
}
func (c *Cline) Models() []string {
@@ -106,7 +87,7 @@ func (c *Cline) Models() []string {
return nil
}
config, err := readJSONFile(filepath.Join(home, ".cline", "data", "globalState.json"))
config, err := fileutil.ReadJSON(filepath.Join(home, ".cline", "data", "globalState.json"))
if err != nil {
return nil
}
@@ -1,4 +1,4 @@
package config
package launch
import (
"encoding/json"
+148
View File
@@ -0,0 +1,148 @@
package launch
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/ollama/ollama/envconfig"
"golang.org/x/mod/semver"
)
// Codex implements Runner for Codex integration
type Codex struct{}
func (c *Codex) String() string { return "Codex" }
const codexProfileName = "ollama-launch"
func (c *Codex) args(model string, extra []string) []string {
args := []string{"--profile", codexProfileName}
if model != "" {
args = append(args, "-m", model)
}
args = append(args, extra...)
return args
}
func (c *Codex) Run(model string, args []string) error {
if err := checkCodexVersion(); err != nil {
return err
}
if err := ensureCodexConfig(); err != nil {
return fmt.Errorf("failed to configure codex: %w", err)
}
cmd := exec.Command("codex", c.args(model, args)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Env = append(os.Environ(),
"OPENAI_API_KEY=ollama",
)
return cmd.Run()
}
// ensureCodexConfig writes a [profiles.ollama-launch] section to ~/.codex/config.toml
// with openai_base_url pointing to the local Ollama server.
func ensureCodexConfig() error {
home, err := os.UserHomeDir()
if err != nil {
return err
}
codexDir := filepath.Join(home, ".codex")
if err := os.MkdirAll(codexDir, 0o755); err != nil {
return err
}
configPath := filepath.Join(codexDir, "config.toml")
return writeCodexProfile(configPath)
}
// writeCodexProfile ensures ~/.codex/config.toml has the ollama-launch profile
// and model provider sections with the correct base URL.
func writeCodexProfile(configPath string) error {
baseURL := envconfig.Host().String() + "/v1/"
sections := []struct {
header string
lines []string
}{
{
header: fmt.Sprintf("[profiles.%s]", codexProfileName),
lines: []string{
fmt.Sprintf("openai_base_url = %q", baseURL),
`forced_login_method = "api"`,
fmt.Sprintf("model_provider = %q", codexProfileName),
},
},
{
header: fmt.Sprintf("[model_providers.%s]", codexProfileName),
lines: []string{
`name = "Ollama"`,
fmt.Sprintf("base_url = %q", baseURL),
},
},
}
content, readErr := os.ReadFile(configPath)
text := ""
if readErr == nil {
text = string(content)
}
for _, s := range sections {
block := strings.Join(append([]string{s.header}, s.lines...), "\n") + "\n"
if idx := strings.Index(text, s.header); idx >= 0 {
// Replace the existing section up to the next section header.
rest := text[idx+len(s.header):]
if endIdx := strings.Index(rest, "\n["); endIdx >= 0 {
text = text[:idx] + block + rest[endIdx+1:]
} else {
text = text[:idx] + block
}
} else {
// Append the section.
if text != "" && !strings.HasSuffix(text, "\n") {
text += "\n"
}
if text != "" {
text += "\n"
}
text += block
}
}
return os.WriteFile(configPath, []byte(text), 0o644)
}
func checkCodexVersion() error {
if _, err := exec.LookPath("codex"); err != nil {
return fmt.Errorf("codex is not installed, install with: npm install -g @openai/codex")
}
out, err := exec.Command("codex", "--version").Output()
if err != nil {
return fmt.Errorf("failed to get codex version: %w", err)
}
// Parse output like "codex-cli 0.87.0"
fields := strings.Fields(strings.TrimSpace(string(out)))
if len(fields) < 2 {
return fmt.Errorf("unexpected codex version output: %s", string(out))
}
version := "v" + fields[len(fields)-1]
minVersion := "v0.81.0"
if semver.Compare(version, minVersion) < 0 {
return fmt.Errorf("codex version %s is too old, minimum required is %s, update with: npm update -g @openai/codex", fields[len(fields)-1], "0.81.0")
}
return nil
}
+229
View File
@@ -0,0 +1,229 @@
package launch
import (
"os"
"path/filepath"
"slices"
"strings"
"testing"
)
func TestCodexArgs(t *testing.T) {
c := &Codex{}
tests := []struct {
name string
model string
args []string
want []string
}{
{"with model", "llama3.2", nil, []string{"--profile", "ollama-launch", "-m", "llama3.2"}},
{"empty model", "", nil, []string{"--profile", "ollama-launch"}},
{"with model and extra args", "qwen3.5", []string{"-p", "myprofile"}, []string{"--profile", "ollama-launch", "-m", "qwen3.5", "-p", "myprofile"}},
{"with sandbox flag", "llama3.2", []string{"--sandbox", "workspace-write"}, []string{"--profile", "ollama-launch", "-m", "llama3.2", "--sandbox", "workspace-write"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := c.args(tt.model, tt.args)
if !slices.Equal(got, tt.want) {
t.Errorf("args(%q, %v) = %v, want %v", tt.model, tt.args, got, tt.want)
}
})
}
}
func TestWriteCodexProfile(t *testing.T) {
t.Run("creates new file when none exists", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
if !strings.Contains(content, "openai_base_url") {
t.Error("missing openai_base_url key")
}
if !strings.Contains(content, "/v1/") {
t.Error("missing /v1/ suffix in base URL")
}
if !strings.Contains(content, `forced_login_method = "api"`) {
t.Error("missing forced_login_method key")
}
if !strings.Contains(content, `model_provider = "ollama-launch"`) {
t.Error("missing model_provider key")
}
if !strings.Contains(content, "[model_providers.ollama-launch]") {
t.Error("missing [model_providers.ollama-launch] section")
}
if !strings.Contains(content, `name = "Ollama"`) {
t.Error("missing model provider name")
}
})
t.Run("appends profile to existing file without profile", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[some_other_section]\nkey = \"value\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if !strings.Contains(content, "[some_other_section]") {
t.Error("existing section was removed")
}
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
})
t.Run("replaces existing profile section", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[profiles.ollama-launch]\nopenai_base_url = \"http://old:1234/v1/\"\n\n[model_providers.ollama-launch]\nname = \"Ollama\"\nbase_url = \"http://old:1234/v1/\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if strings.Contains(content, "old:1234") {
t.Error("old URL was not replaced")
}
if strings.Count(content, "[profiles.ollama-launch]") != 1 {
t.Errorf("expected exactly one [profiles.ollama-launch] section, got %d", strings.Count(content, "[profiles.ollama-launch]"))
}
if strings.Count(content, "[model_providers.ollama-launch]") != 1 {
t.Errorf("expected exactly one [model_providers.ollama-launch] section, got %d", strings.Count(content, "[model_providers.ollama-launch]"))
}
})
t.Run("replaces profile while preserving following sections", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[profiles.ollama-launch]\nopenai_base_url = \"http://old:1234/v1/\"\n[another_section]\nfoo = \"bar\"\n"
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if strings.Contains(content, "old:1234") {
t.Error("old URL was not replaced")
}
if !strings.Contains(content, "[another_section]") {
t.Error("following section was removed")
}
if !strings.Contains(content, "foo = \"bar\"") {
t.Error("following section content was removed")
}
})
t.Run("appends newline to file not ending with newline", func(t *testing.T) {
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
existing := "[other]\nkey = \"val\""
os.WriteFile(configPath, []byte(existing), 0o644)
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
// Should not have double blank lines from missing trailing newline
if strings.Contains(content, "\n\n\n") {
t.Error("unexpected triple newline in output")
}
})
t.Run("uses custom OLLAMA_HOST", func(t *testing.T) {
t.Setenv("OLLAMA_HOST", "http://myhost:9999")
tmpDir := t.TempDir()
configPath := filepath.Join(tmpDir, "config.toml")
if err := writeCodexProfile(configPath); err != nil {
t.Fatal(err)
}
data, _ := os.ReadFile(configPath)
content := string(data)
if !strings.Contains(content, "myhost:9999/v1/") {
t.Errorf("expected custom host in URL, got:\n%s", content)
}
})
}
func TestEnsureCodexConfig(t *testing.T) {
t.Run("creates .codex dir and config.toml", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if err := ensureCodexConfig(); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatalf("config.toml not created: %v", err)
}
content := string(data)
if !strings.Contains(content, "[profiles.ollama-launch]") {
t.Error("missing [profiles.ollama-launch] header")
}
if !strings.Contains(content, "openai_base_url") {
t.Error("missing openai_base_url key")
}
})
t.Run("is idempotent", func(t *testing.T) {
tmpDir := t.TempDir()
setTestHome(t, tmpDir)
if err := ensureCodexConfig(); err != nil {
t.Fatal(err)
}
if err := ensureCodexConfig(); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(tmpDir, ".codex", "config.toml")
data, _ := os.ReadFile(configPath)
content := string(data)
if strings.Count(content, "[profiles.ollama-launch]") != 1 {
t.Errorf("expected exactly one [profiles.ollama-launch] section after two calls, got %d", strings.Count(content, "[profiles.ollama-launch]"))
}
if strings.Count(content, "[model_providers.ollama-launch]") != 1 {
t.Errorf("expected exactly one [model_providers.ollama-launch] section after two calls, got %d", strings.Count(content, "[model_providers.ollama-launch]"))
}
})
}
+664
View File
@@ -0,0 +1,664 @@
package launch
import (
"bytes"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/ollama/ollama/cmd/config"
"github.com/spf13/cobra"
)
func captureStderr(t *testing.T, fn func()) string {
t.Helper()
oldStderr := os.Stderr
r, w, err := os.Pipe()
if err != nil {
t.Fatalf("failed to create stderr pipe: %v", err)
}
os.Stderr = w
defer func() {
os.Stderr = oldStderr
}()
done := make(chan string, 1)
go func() {
var buf bytes.Buffer
_, _ = io.Copy(&buf, r)
done <- buf.String()
}()
fn()
_ = w.Close()
return <-done
}
func TestLaunchCmd(t *testing.T) {
mockCheck := func(cmd *cobra.Command, args []string) error {
return nil
}
mockTUI := func(cmd *cobra.Command) {}
cmd := LaunchCmd(mockCheck, mockTUI)
t.Run("command structure", func(t *testing.T) {
if cmd.Use != "launch [INTEGRATION] [-- [EXTRA_ARGS...]]" {
t.Errorf("Use = %q, want %q", cmd.Use, "launch [INTEGRATION] [-- [EXTRA_ARGS...]]")
}
if cmd.Short == "" {
t.Error("Short description should not be empty")
}
if cmd.Long == "" {
t.Error("Long description should not be empty")
}
if !strings.Contains(cmd.Long, "hermes") {
t.Error("Long description should mention hermes")
}
if !strings.Contains(cmd.Long, "kimi") {
t.Error("Long description should mention kimi")
}
})
t.Run("flags exist", func(t *testing.T) {
if cmd.Flags().Lookup("model") == nil {
t.Error("--model flag should exist")
}
if cmd.Flags().Lookup("config") == nil {
t.Error("--config flag should exist")
}
if cmd.Flags().Lookup("restore") == nil {
t.Error("--restore flag should exist")
}
if cmd.Flags().Lookup("yes") == nil {
t.Error("--yes flag should exist")
}
})
t.Run("PreRunE is set", func(t *testing.T) {
if cmd.PreRunE == nil {
t.Error("PreRunE should be set to checkServerHeartbeat")
}
})
}
func TestLaunchCmdTUICallback(t *testing.T) {
mockCheck := func(cmd *cobra.Command, args []string) error {
return nil
}
t.Run("no args calls TUI", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{})
_ = cmd.Execute()
if !tuiCalled {
t.Error("TUI callback should be called when no args provided")
}
})
t.Run("integration arg bypasses TUI", func(t *testing.T) {
srv := httptest.NewServer(http.NotFoundHandler())
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"claude"})
_ = cmd.Execute()
if tuiCalled {
t.Error("TUI callback should NOT be called when integration arg provided")
}
})
t.Run("--model flag without integration returns error", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"--model", "test-model"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected --model without an integration to fail")
}
if !strings.Contains(err.Error(), "require an integration name") {
t.Fatalf("expected integration-name guidance, got %v", err)
}
if tuiCalled {
t.Error("TUI callback should NOT be called when --model is provided without an integration")
}
})
t.Run("--config flag without integration returns error", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"--config"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected --config without an integration to fail")
}
if !strings.Contains(err.Error(), "require an integration name") {
t.Fatalf("expected integration-name guidance, got %v", err)
}
if tuiCalled {
t.Error("TUI callback should NOT be called when --config is provided without an integration")
}
})
t.Run("--yes flag without integration returns error", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"--yes"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected --yes without an integration to fail")
}
if !strings.Contains(err.Error(), "require an integration name") {
t.Fatalf("expected integration-name guidance, got %v", err)
}
if tuiCalled {
t.Error("TUI callback should NOT be called when --yes is provided without an integration")
}
})
t.Run("extra args without integration return error", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"--model", "test-model", "--", "--sandbox", "workspace-write"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected flags and extra args without an integration to fail")
}
if !strings.Contains(err.Error(), "require an integration name") {
t.Fatalf("expected integration-name guidance, got %v", err)
}
if tuiCalled {
t.Error("TUI callback should NOT be called when flags or extra args are provided without an integration")
}
})
t.Run("--restore flag without integration returns error", func(t *testing.T) {
tuiCalled := false
mockTUI := func(cmd *cobra.Command) {
tuiCalled = true
}
cmd := LaunchCmd(mockCheck, mockTUI)
cmd.SetArgs([]string{"--restore"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected --restore without an integration to fail")
}
if !strings.Contains(err.Error(), "require an integration name") {
t.Fatalf("expected integration-name guidance, got %v", err)
}
if tuiCalled {
t.Error("TUI callback should NOT be called when --restore is provided without an integration")
}
})
}
func TestLaunchCmdNilHeartbeat(t *testing.T) {
cmd := LaunchCmd(nil, nil)
if cmd == nil {
t.Fatal("LaunchCmd returned nil")
}
if cmd.PreRunE != nil {
t.Log("Note: PreRunE is set even when nil is passed (acceptable)")
}
}
func TestLaunchCmdModelFlagFiltersDisabledCloudFromSavedConfig(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
if err := config.SaveIntegration("stubeditor", []string{"glm-5:cloud"}); err != nil {
t.Fatalf("failed to seed saved config: %v", err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/status":
fmt.Fprintf(w, `{"cloud":{"disabled":true,"source":"config"}}`)
case "/api/show":
fmt.Fprintf(w, `{"model":"llama3.2"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherEditorRunner{}
restore := OverrideIntegration("stubeditor", stub)
defer restore()
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command failed: %v", err)
}
saved, err := config.LoadIntegration("stubeditor")
if err != nil {
t.Fatalf("failed to reload integration config: %v", err)
}
if diff := cmp.Diff([]string{"llama3.2"}, saved.Models); diff != "" {
t.Fatalf("saved models mismatch (-want +got):\n%s", diff)
}
if diff := cmp.Diff([][]string{{"llama3.2"}}, stub.edited); diff != "" {
t.Fatalf("editor models mismatch (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run with llama3.2, got %q", stub.ranModel)
}
}
func TestLaunchCmdModelFlagClearsDisabledCloudOverride(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/status":
fmt.Fprintf(w, `{"cloud":{"disabled":true,"source":"config"}}`)
case "/api/experimental/model-recommendations":
fmt.Fprint(w, `{"recommendations":[]}`)
case "/api/tags":
fmt.Fprint(w, `{"models":[{"name":"llama3.2"}]}`)
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherSingleRunner{}
restore := OverrideIntegration("stubapp", stub)
defer restore()
oldSelector := DefaultSingleSelector
defer func() { DefaultSingleSelector = oldSelector }()
var selectorCalls int
var gotCurrent string
DefaultSingleSelector = func(title string, items []SelectionItem, current string) (string, error) {
selectorCalls++
gotCurrent = current
return "llama3.2", nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubapp", "--model", "glm-5:cloud"})
stderr := captureStderr(t, func() {
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command failed: %v", err)
}
})
if selectorCalls != 1 {
t.Fatalf("expected disabled cloud override to fall back to selector, got %d calls", selectorCalls)
}
if gotCurrent != "" {
t.Fatalf("expected disabled override to be cleared before selection, got current %q", gotCurrent)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run with replacement local model, got %q", stub.ranModel)
}
if !strings.Contains(stderr, "Warning: ignoring --model glm-5:cloud because cloud is disabled") {
t.Fatalf("expected disabled-cloud warning, got stderr: %q", stderr)
}
saved, err := config.LoadIntegration("stubapp")
if err != nil {
t.Fatalf("failed to reload integration config: %v", err)
}
if diff := cmp.Diff([]string{"llama3.2"}, saved.Models); diff != "" {
t.Fatalf("saved models mismatch (-want +got):\n%s", diff)
}
}
func TestLaunchCmdAutodiscoveryDefaultLaunchDoesNotForceConfigure(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withInteractiveSession(t, true)
withLauncherHooks(t)
runner := &launcherManagedAutodiscoveryRunner{
autodiscoveryConfigured: true,
}
restore := OverrideIntegration("stubauto", runner)
defer restore()
if err := config.SaveIntegration("stubauto", []string{"Ollama Cloud"}); err != nil {
t.Fatalf("failed to save managed integration config: %v", err)
}
if err := config.MarkIntegrationOnboarded("stubauto"); err != nil {
t.Fatalf("failed to mark integration onboarded: %v", err)
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {
t.Fatal("TUI callback should not run for direct integration launch")
})
cmd.SetArgs([]string{"stubauto"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command failed: %v", err)
}
if runner.autodiscoveryConfigures != 0 {
t.Fatalf("expected default autodiscovery launch to reuse existing config, got %d configures", runner.autodiscoveryConfigures)
}
if runner.ranModel != "Ollama Cloud" {
t.Fatalf("expected launch to run autodiscovery label, got %q", runner.ranModel)
}
}
func TestLaunchCmdYes_AutoConfirmsLaunchPromptPath(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, false)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
case "/api/status":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"not found"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherEditorRunner{paths: []string{"/tmp/stubeditor.json"}}
restore := OverrideIntegration("stubeditor", stub)
defer restore()
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("unexpected prompt with --yes: %q", prompt)
return false, nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2", "--yes"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command with --yes failed: %v", err)
}
if diff := cmp.Diff([][]string{{"llama3.2"}}, stub.edited); diff != "" {
t.Fatalf("editor models mismatch (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run with llama3.2, got %q", stub.ranModel)
}
}
func TestLaunchCmdHeadlessWithYes_AutoPullsMissingLocalModel(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, false)
var pullCalled bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"model not found"}`)
case "/api/pull":
pullCalled = true
w.WriteHeader(http.StatusOK)
fmt.Fprint(w, `{"status":"success"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherSingleRunner{}
restore := OverrideIntegration("stubapp", stub)
defer restore()
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("unexpected prompt with --yes in headless autopull path: %q", prompt)
return false, nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubapp", "--model", "missing-model", "--yes"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command with --yes failed: %v", err)
}
if !pullCalled {
t.Fatal("expected missing local model to be auto-pulled with --yes in headless mode")
}
if stub.ranModel != "missing-model" {
t.Fatalf("expected launch to run with pulled model, got %q", stub.ranModel)
}
}
func TestLaunchCmdHeadlessWithoutYes_AllowsConfiguredLaunch(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, false)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
case "/api/status":
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"error":"not found"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherEditorRunner{paths: []string{"/tmp/stubeditor.json"}}
restore := OverrideIntegration("stubeditor", stub)
defer restore()
DefaultConfirmPrompt = func(prompt string, options ConfirmOptions) (bool, error) {
t.Fatalf("unexpected prompt in headless non-yes mode: %q", prompt)
return false, nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubeditor", "--model", "llama3.2"})
err := cmd.Execute()
if err != nil {
t.Fatalf("expected launch command to succeed without --yes when an explicit model is provided, got %v", err)
}
if diff := compareStringSlices(stub.edited, [][]string{{"llama3.2"}}); diff != "" {
t.Fatalf("unexpected editor writes (-want +got):\n%s", diff)
}
if stub.ranModel != "llama3.2" {
t.Fatalf("expected launch to run configured model, got %q", stub.ranModel)
}
}
func TestLaunchCmdIntegrationArgPromptsForModelWithSavedSelection(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
if err := config.SaveIntegration("stubapp", []string{"llama3.2"}); err != nil {
t.Fatalf("failed to seed saved config: %v", err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/experimental/model-recommendations":
fmt.Fprint(w, `{"recommendations":[]}`)
case "/api/tags":
fmt.Fprint(w, `{"models":[{"name":"llama3.2"},{"name":"qwen3:8b"}]}`)
case "/api/show":
fmt.Fprint(w, `{"model":"qwen3:8b"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherSingleRunner{}
restore := OverrideIntegration("stubapp", stub)
defer restore()
oldSelector := DefaultSingleSelector
defer func() { DefaultSingleSelector = oldSelector }()
var gotCurrent string
DefaultSingleSelector = func(title string, items []SelectionItem, current string) (string, error) {
gotCurrent = current
return "qwen3:8b", nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubapp"})
if err := cmd.Execute(); err != nil {
t.Fatalf("launch command failed: %v", err)
}
if gotCurrent != "llama3.2" {
t.Fatalf("expected selector current model to be saved model llama3.2, got %q", gotCurrent)
}
if stub.ranModel != "qwen3:8b" {
t.Fatalf("expected launch to run selected model qwen3:8b, got %q", stub.ranModel)
}
saved, err := config.LoadIntegration("stubapp")
if err != nil {
t.Fatalf("failed to reload integration config: %v", err)
}
if diff := cmp.Diff([]string{"qwen3:8b"}, saved.Models); diff != "" {
t.Fatalf("saved models mismatch (-want +got):\n%s", diff)
}
}
func TestLaunchCmdHeadlessYes_IntegrationRequiresModelEvenWhenSaved(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, false)
if err := config.SaveIntegration("stubapp", []string{"llama3.2"}); err != nil {
t.Fatalf("failed to seed saved config: %v", err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/show":
fmt.Fprint(w, `{"model":"llama3.2"}`)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherSingleRunner{}
restore := OverrideIntegration("stubapp", stub)
defer restore()
oldSelector := DefaultSingleSelector
defer func() { DefaultSingleSelector = oldSelector }()
DefaultSingleSelector = func(title string, items []SelectionItem, current string) (string, error) {
t.Fatal("selector should not be called for headless --yes saved-model launch")
return "", nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubapp", "--yes"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected launch command to fail when --yes is used headlessly without --model")
}
if !strings.Contains(err.Error(), "requires --model <model>") {
t.Fatalf("expected actionable --model guidance, got %v", err)
}
if stub.ranModel != "" {
t.Fatalf("expected launch to abort before run, got %q", stub.ranModel)
}
}
func TestLaunchCmdHeadlessYes_IntegrationWithoutSavedModelReturnsError(t *testing.T) {
tmpDir := t.TempDir()
setLaunchTestHome(t, tmpDir)
withLauncherHooks(t)
withInteractiveSession(t, false)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
t.Setenv("OLLAMA_HOST", srv.URL)
stub := &launcherSingleRunner{}
restore := OverrideIntegration("stubapp", stub)
defer restore()
oldSelector := DefaultSingleSelector
defer func() { DefaultSingleSelector = oldSelector }()
DefaultSingleSelector = func(title string, items []SelectionItem, current string) (string, error) {
t.Fatal("selector should not be called for headless --yes without saved model")
return "", nil
}
cmd := LaunchCmd(func(cmd *cobra.Command, args []string) error { return nil }, func(cmd *cobra.Command) {})
cmd.SetArgs([]string{"stubapp", "--yes"})
err := cmd.Execute()
if err == nil {
t.Fatal("expected launch command to fail when --yes is used headlessly without --model")
}
if !strings.Contains(err.Error(), "requires --model <model>") {
t.Fatalf("expected actionable --model guidance, got %v", err)
}
if stub.ranModel != "" {
t.Fatalf("expected launch to abort before run, got %q", stub.ranModel)
}
}
Loaded 100 of 1413 files, more files were not shown because too many files have changed in this diff. Show more