ollama/docs/development.md

Development

Install prerequisites:

• Go
• CMake 3.24 or newer
• C/C++ compiler: Clang on macOS, Visual Studio 2022 C++ tools on Windows, or GCC/Clang on Linux
• Ninja in PATH is recommended, especially on Windows

For pure Go iteration against an existing native payload, run Ollama from the repository root:

go run . serve

Note

Ollama includes native code compiled with CGO. From time to time these data structures can change and CGO can get out of sync resulting in unexpected crashes. You can force a full build of the native code by running go clean -cache first.

Native build model

For a fresh checkout, or after changing native code, build from the repository root. On macOS arm64, this builds Metal inference. On all other platforms this builds CPU-only inference. It builds the Go binary at the repository root and installs the native runtime payload under build/lib/ollama.

cmake -B build .
cmake --build build --parallel 8
./ollama serve

To install into a standard prefix layout:

cmake --install build --prefix /path/to/install

On all platforms except macOS arm64, to build GPU backends select the backends explicitly:

cmake -B build . -DOLLAMA_LLAMA_BACKENDS="cuda_v13;vulkan"
cmake --build build --parallel 8

Supported backend values are cuda_v12, cuda_v13, rocm_v7_1, rocm_v7_2, vulkan, cuda_jetpack5, and cuda_jetpack6.

Use standard CMake architecture overrides to narrow GPU builds for local hardware:

# CUDA
cmake -B build . -DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=native

# ROCm / HIP
cmake -B build . -DOLLAMA_LLAMA_BACKENDS=rocm_v7_2 -DCMAKE_HIP_ARCHITECTURES=gfx1100

You can tune GGML build options by setting GGML_* values during configure. For example, to build CUDA v12 for Pascal without flash attention kernels:

cmake -B build . -DOLLAMA_LLAMA_BACKENDS=cuda_v12 -DCMAKE_CUDA_ARCHITECTURES=61 -DGGML_CUDA_FA=OFF

macOS (Apple Silicon)

Additional prerequisites:

MLX Metal requires the Metal toolchain. Install Xcode first, then:

xcodebuild -downloadComponent MetalToolchain

Windows

Additional prerequisites:

• Visual Studio 2022 including the Native Desktop Workload
• (Optional) AMD GPU support
  • ROCm
• (Optional) NVIDIA GPU support
  • CUDA SDK
• (Optional) Vulkan GPU support
  • Vulkan SDK - useful for AMD/Intel GPUs
• (Optional) MLX engine support
  • CUDA 13+ SDK
  • cuDNN 9+

For Ninja builds, run CMake from a Developer PowerShell/Command Prompt or another shell where the Visual Studio compiler is available.

Building for Vulkan requires VULKAN_SDK environment variable:

PowerShell

$env:VULKAN_SDK="C:\VulkanSDK\<version>"

CMD

set VULKAN_SDK=C:\VulkanSDK\<version>

Windows (ARM)

Windows ARM does not support additional acceleration libraries at this time.

Linux

Additional prerequisites:

• (Optional) AMD GPU support
  • ROCm
• (Optional) NVIDIA GPU support
  • CUDA SDK
• (Optional) Vulkan GPU support
  • Vulkan SDK - useful for AMD/Intel GPUs
  • Or install via package manager: sudo apt install vulkan-sdk (Ubuntu/Debian) or sudo dnf install vulkan-sdk (Fedora/CentOS)
• (Optional) MLX engine support
  • CUDA 13+ SDK
  • cuDNN 9+
  • OpenBLAS/LAPACK: sudo apt install libopenblas-dev liblapack-dev liblapacke-dev (Ubuntu/Debian)

Important

Ensure prerequisites are in PATH before running CMake.

MLX Engine (Optional)

The MLX engine enables running safetensor based models. On macOS arm64, MLX is enabled by default. On other platforms, MLX backends are selected with OLLAMA_MLX_BACKENDS.

CUDA

Requires CUDA 13+ and cuDNN 9+.

cmake -B build . -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build --parallel 8

Local MLX source overrides

To build against a local checkout of MLX and/or MLX-C (useful for development), set environment variables before running CMake:

export OLLAMA_MLX_SOURCE=/path/to/mlx
export OLLAMA_MLX_C_SOURCE=/path/to/mlx-c

On macOS arm64:

OLLAMA_MLX_SOURCE=../mlx OLLAMA_MLX_C_SOURCE=../mlx-c cmake -B build .
cmake --build build --parallel 8

For CUDA:

$env:OLLAMA_MLX_SOURCE="../mlx"
$env:OLLAMA_MLX_C_SOURCE="../mlx-c"
cmake -B build . -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build --parallel 8

Docker

docker build .

ROCm

docker build --build-arg FLAVOR=rocm .

Running tests

To run tests, use go test:

go test ./...

Library detection

Ollama looks for native helper binaries and acceleration libraries in installed and local development layouts:

• ../lib/ollama for standard installs where ollama is under bin/
• ./lib/ollama for Windows release-style payloads and local dist output
• . for macOS release artifacts that colocate helpers with ollama
• build/lib/ollama and dist/<platform>/lib/ollama for local development builds

If the libraries are not found, Ollama will not run with any acceleration libraries.