Since #2087 moved mlx, mlx-lm, mlx-vlm and mflux out of `dependencies`
into the `mlx` extra, the documented setup path never installs them, and
`[tool.uv]` sets no default extras. `uv run exo` starts the API, then
every runner crashes with `ModuleNotFoundError: No module named 'mlx'`
(#2156).
Adds the extra to the commands documented as the way to set up:
- README macOS: `uv sync --extra mlx`
- README Linux: `uv sync --extra mlx-cpu`, with the mlx-cuda12 /
mlx-cuda13 alternatives
- CONTRIBUTING.md quick start
- `just sync` / `just sync-clean`, which `just build-app` runs before
pyinstaller — the spec aborts when the mlx Metal libraries are missing
Docs plus two justfile recipes; no dependency or lock changes. Backend
choice stays explicit, since mlx-cpu / mlx-cuda12 / mlx-cuda13 /
mlx-none are declared as conflicting extras.
#2234 overlaps on the macOS README block only, via a setup script that
runs `uv sync --extra mlx`; the Linux block, CONTRIBUTING.md and the
justfile are not covered there.
Adds a model card for
[moonshotai/Kimi-K2.7-Code](https://huggingface.co/moonshotai/Kimi-K2.7-Code),
released 2026-06-12.
Same architecture as Kimi K2.6 (`kimi_k25`, 61 layers, official INT4),
so the card mirrors the existing `moonshotai--Kimi-K2.6.toml`. Sampling
defaults per the model card (temperature 1.0 / top_p 0.95 for thinking
mode).
**Vision:** the official repo ships MoonViT weights inline, so I
extracted the 335 `vision_tower.*` / `mm_projector.*` tensors
(unmodified bf16) into
[aidiffuser/Kimi-K2.7-Code-vision](https://huggingface.co/aidiffuser/Kimi-K2.7-Code-vision),
following the `exolabs/Kimi-K2.6-vision` format. The vision config is
byte-identical to K2.6's; the extraction script is included in the repo
for verification. Happy to have this re-hosted under the exolabs org if
you prefer — it's a one-line change to the card.
**Tested:** distributed serving on 2× Mac Studio M3 Ultra (512 GB),
tensor parallelism, text + thinking + image understanding all confirmed
working.
Co-authored-by: aidiffuser <your-noreply-email@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Summary
Upgrade devalue from 5.5.0 to 5.6.2 to fix CVE-2026-22774.
## Vulnerability
| Field | Value |
|-------|-------|
| **ID** | CVE-2026-22774 |
| **Severity** | HIGH |
| **Scanner** | trivy |
| **Rule** | `CVE-2026-22774` |
| **File** | `dashboard/package-lock.json` |
| **Assessment** | Likely exploitable |
**Description**: devalue: devalue: Denial of Service due to excessive
resource consumption from untrusted input
## Evidence
**Scanner confirmation**: trivy rule `CVE-2026-22774` flagged this
pattern.
**Production code**: This file is in the production codebase, not
test-only code.
## Threat Model Context
This is a web service - vulnerabilities in request handlers are directly
exploitable by remote attackers.
## Changes
- `dashboard/package.json`
- `dashboard/package-lock.json`
## Verification
- [x] Build passes
- [x] Scanner re-scan confirms fix
- [x] LLM code review passed
---
*This change addresses a pattern flagged by static analysis. The code
path handles user-influenced input and the fix reduces the attack
surface against both manual and automated exploitation.*
---
*Automated security fix by [OrbisAI Security](https://orbisappsec.com)*
## Motivation
exo is now available as a Homebrew cask, so the README should show the
simplest macOS installation path alongside the existing DMG download.
Fixes https://github.com/exo-explore/exo/issues/2105https://github.com/exo-explore/exo/issues/176
## Changes
- Added `brew install --cask exo` to the macOS App section of
`README.md`
- Kept the existing DMG download link as the first installation option
## Why It Works
Adding the Homebrew cask command gives macOS users a
package-manager-managed installation path while preserving the existing
DMG download option.
## Test Plan
### Manual Testing
- Reviewed the rendered Markdown structure in `README.md`
### Automated Testing
- Not run. Documentation-only change.
## Related
- https://github.com/Homebrew/homebrew-cask/pull/265956
## Motivation
(I think it) Makes Evan's massive PR easier to merge later on
## Changes
- Renamed exo_pyo3_bindings to exo_rs
- Upgraded versions of pyo3-based dependencies
- Renamed PyFromSwarm to just FromSwarm, and PyNetworkingHandle to just
NetworkingHandle
## Motivation
Energy was reported as a single aggregate. Split into prefill vs.
generation so each phase can be analysed independently.
## Changes
- `PowerSampler`: `mark_prefill_done()` + `trapezoidal_energy_range()`
helper; `result()` now emits per-phase splits.
- `PowerUsage` / `NodePowerStats`: optional `prefill_*` / `generation_*`
fields (back-compat: `None` if unmarked).
- API marks the boundary on the first non-`PrefillProgressChunk`.
- `bench/exo_bench.py` surfaces the split in the log line and persists
`power_usage` to JSON.
- METHODOLOGY: one sentence + one bullet.
## Why It Works
First non-prefill chunk *is* the boundary. Anchoring a sample there and
interpolating power at the boundary makes phase energies sum exactly to
the unsplit total.
## Test Plan
### Manual Testing
`eco`-reserved nodes:
- M3 Ultra, Qwen3-VL-4B, pp=8192/tg=1024: server 1940 J vs client 1931 J
(+0.5 %)
- M4 Pro, Qwen3.6-27B, pp=16384/tg=2048: server 20,292 J vs client
20,221 J (+0.35 %)
### Automated Testing
5 new tests in `test_power_sampler.py` (range integrator,
splits-sum-to-total, `None`-when-unmarked, idempotency). 14/14 pass.
## Motivation
Partially fixes [this](https://github.com/exo-explore/exo/issues/2098)
issue. Removed erroneous logic for telling user to download when they
already downloaded.
Could not figure out about the "spontaneous crashes" in that issue,
author should consolidate more logs and open a new issue dedicated to
that. I believe
[this](https://github.com/exo-explore/exo/commit/74e9fe15e62fe189dc7e019db86e75c83eca2721)
commit solved some EventRouter-related crashes, which was mentioned in
[this](https://github.com/exo-explore/exo/issues/2098) issue, so it may
have already been solved. If not, should be re-submitted as a new issue.
## Changes
- Consolidated _resolve_and_validate_text_model and
_validate_image_model into one function: _validate_model_has_instance;
- + They already had virtually identical logic, it being different seems
to be an artifact of history
- + Added logic to ensure that _trigger_notify_user_to_download_model is
only called when no such model is downloaded, not just if there is no
instance of it
- Added a new `/instance/await` SSE streaming endpoint to wait for when
a model has an instance available. Complements instance-placement API,
so we can wait till that is done without client-side polling.
- Updated docs and a /tmp script to reflect some of the changes
- Updated dashboard `getModelForRequest` to only return model ID if an
instance exists for it, and updated bits to use `handleChatSend` instead
of `sendMessage` because that checks for if a model instance exists
first.
## Why It Works
The problem was that there was erroneous logging for model not
downloaded. I fixed that logic. The rest is extra.
## Motivation
Addresses [this](https://github.com/exo-explore/exo/issues/1931) issue.
## Changes
You can now launch Exo as a legacy SysV-style daemin (in the background)
with `--legacy-daemon` flag.
NOTE: don't use it if you're managing Exo with systemd or launchd
SIDE FIX: the macmon process not found trace is no longer displayed on
process shutdown via ctrl+c, that error is supressed.
## Why It Works
Because I used a daemonization library and tweaked it not to break
multiprocessing.
## Test Plan
I ran it in daemon mode, non daemon mode, etc., and pid locking +
inference + everything else works just fine.
Also ran it `ssh user@host -t 'cd exo && nohup nix run .#exo --
--legacy-daemon'` on a 4-node TB mac-mini cluster and the mDNS didn't
die
## Motivation
Trying to (partially) fix
[this](https://github.com/exo-explore/exo/issues/2101) issue.
## Changes
Changed channels (in channels.py) to support exception overriding.
Made EventRouter channels throw a subclass of the resource closed/broken
errors.
The current lifetime logic of EventRouter in event loop no longer blows
up because components that use channels from EventRouter now catch the
subclass exceptions in the run method: Worker, Master,
DownloadCoordinator, RunnerSupervisor.
Added logic to throw when API server exits without being asked to shut
down - this kill the sleep-forever in the task-group.
# Runner error handling
## Motivation
Runner failures were mostly surfaced as plain shutdown messages, which
made root cause hard to spot from API errors or runner status.
This adds a MVP path for preserving runner crash context and attaching
known stderr diagnostics to failure reports.
## Changes
- Added `RunnerTerminationError` for Python exceptions raised inside
runner bootstrap
- Changed runner bootstrap to send `Event | RunnerTerminationError` over
the private runner channel
- Moved public `RunnerFailed` emission back into supervisor
- Added stderr-only `RunnerDiagnosticCollector`
- + Added known diagnostics for Metal GPU timeout, ring socket receive
errno, and ring transport abort
- Added diagnostics to `RunnerFailed` and `ErrorChunk`
- Tweaked async process termination to join briefly before
terminate/kill
- Updated tests/fixtures for new failure payload shape
- Added Ruff VS Code formatter settings
## Why It Works
Runner child now reports raw-ish failure context to supervisor instead
of publishing failed status directly.
Supervisor still owns process lifecycle, exit code/signal handling,
in-flight task error chunks, and final runner status. Stderr diagnostics
stay best effort and only known root-cause variants are surfaced.
## Test Plan
### Manual Testing
Hardware: remote runner logs from e16/e11/e4/e2
What you did:
- inspected live runner stderr logs
- used observed Metal GPU timeout and ring socket errors as initial
diagnostic targets
### Automated Testing
- `nix flake check`
- supervisor test covers error chunk + failed status emission
- plan lifecycle test updated for failed runner diagnostics
- type/lint checks cover new runner channel union
---------
Co-authored-by: Evan Quiney <evanev7@gmail.com>
## Motivation
Streaming /v1/chat/completions responses emitted null for tool_calls,
function_call, name, and tool_call_id in every delta chunk. The OpenAI
streaming spec marks these fields as non-nullable — they must either
carry a
real value or be absent entirely. Spec-correct clients doing
delta.get("tool_calls", []) receive None and crash with 'NoneType'
object is
not iterable.
Root cause: the streaming serialisation path called model_dump_json()
without
exclude_none=True, while the request-parsing path already used it
correctly.
Three call sites in chat_completions.py and two in responses.py were
affected.
## Testing
Before — every delta carries explicit nulls:
$ curl -sN -X POST http://localhost:52415/v1/chat/completions \
-H 'Content-Type: application/json' \
-d
'{"model":"mlx-community/Qwen3.5-2B-MLX-8bit","messages":[{"role":"user","
content":"hi"}],"max_tokens":3,"stream":true}' \
| grep "^data: "
data:
{"id":"7c4dae10-...","choices":[{"index":0,"delta":{"role":"assistant","c
ontent":null,"reasoning_content":"Okay","name":null,"tool_calls":null,"tool_cal
l_id":null,"function_call":null},"logprobs":null,"finish_reason":null,"usage":n
ull}],"usage":null,"service_tier":null}
data:
{"id":"7c4dae10-...","choices":[{"index":0,"delta":{"role":"assistant","c
ontent":null,"reasoning_content":",","name":null,"tool_calls":null,"tool_call_i
d":null,"function_call":null},"logprobs":null,"finish_reason":null,"usage":null
}],"usage":null,"service_tier":null}
data:
{"id":"7c4dae10-...","choices":[{"index":0,"delta":{"role":"assistant","c
ontent":"
the","reasoning_content":null,"name":null,"tool_calls":null,"tool_cal
l_id":null,"function_call":null},"logprobs":null,"finish_reason":"length","usag
e":{"prompt_tokens":11,...}}],"usage":null,"service_tier":null}
data: [DONE]
After — only populated fields are emitted:
data:
{"id":"demo","object":"chat.completion","created":...,"model":"mlx-commun
ity/Qwen3.5-2B-MLX-8bit","choices":[{"index":0,"delta":{"role":"assistant","rea
soning_content":"Okay"}}]}
data:
{"id":"demo","object":"chat.completion","created":...,"model":"mlx-commun
ity/Qwen3.5-2B-MLX-8bit","choices":[{"index":0,"delta":{"role":"assistant","rea
soning_content":","}}]}
data:
{"id":"demo","object":"chat.completion","created":...,"model":"mlx-commun
ity/Qwen3.5-2B-MLX-8bit","choices":[{"index":0,"delta":{"role":"assistant","con
tent":"
the"},"finish_reason":"length"}],"usage":{"prompt_tokens":11,"completio
n_tokens":3,"total_tokens":14,...}}
data: [DONE]
## Motivation
<!-- Why is this change needed? What problem does it solve? -->
<!-- If it fixes an open issue, please link to the issue here -->
When you first run `uv run exo` you get an error like :
`FileNotFoundError: [Errno 2] No such file or directory:
'/Users/heidar/.exo/models'`
Manually tested on Macbook Pro M1 32GB
Fixes issue - https://github.com/exo-explore/exo/issues/2090
## Motivation
We want to use log mining tools like
[Drain3](https://github.com/logpai/Drain3) to get standardized error
formats, but for that we should record runner stdout/stderr in a massive
append-only log to gather training data for such tools. Also useful for
future opt-in telemetry.
## Changes
The stdout/stderr from runner now splits into 3 tasks:
1) raw write to dedicated runner logs
2) sanitized line-by-line logging with log-guru
3) stub for further error-processing (i.e. turning lines into errors)
### Manual Testing
Works on 4x mac mini clusted connected as TB4 ring.
## Motivation
To fix https://github.com/exo-explore/exo/issues/2068
## Changes
Adds queue shutdown logic & hard-timeouts for closing server.
## Why It Works
Prevents API from hanging more than 5 seconds.
## Motivation
GLM 4.7 reuses the GLM 4 chat-template tokenizer, but the model card and
EOS-detection path didn't have an explicit mapping for it, so
OpenAI-compatible clients didn't see a clean stop and the runner emitted
follow-on role turns (e.g. \`<|user|>\` continuations after
\`<|assistant|>\`'s output).
## Changes
\`src/exo/worker/engines/mlx/utils_mlx.py\` — add the GLM 4 stop-token
IDs as the EOS set when the loaded model's tokenizer matches GLM 4 / 4.7
chat templates.
## Why It Works
The GLM 4 tokenizer's \`<|user|>\`, \`<|observation|>\`, and
\`<|endoftext|>\` IDs are stable across the GLM 4 / 4.7 line; treating
any of them as EOS lets the runner stop at the assistant turn boundary
the same way it stops at \`</s>\` for Llama-style models. No
prompt-template changes — only the stop set widens.
## Test Plan
### Automated Testing
New unit test
\`src/exo/worker/tests/unittests/test_mlx/test_eos_token_ids.py\`
covering: GLM 4 / 4.7 path returns the expected stop ID set; non-GLM
path returns the standard EOS only.
\`\`\`
src/exo/worker/tests/unittests/test_mlx/test_eos_token_ids.py ..
=== 2 passed in 0.01s ===
\`\`\`
\`uv run basedpyright\` and \`uv run ruff check\` both clean.
### Manual Testing
Hardware: 4-node Apple Silicon cluster, M5 Max master.
- Loaded \`mlx-community/GLM-4.7-Air-mlx-4bit\`, ran chat completion via
\`/v1/chat/completions\`. Before this fix the assistant turn ran on into
a synthetic \`<|user|>\` continuation; after the fix the response stops
cleanly at the assistant boundary.
---------
Co-authored-by: jw-wcv <101585096+jw-wcv@users.noreply.github.com>
Co-authored-by: Evan Quiney <evanev7@gmail.com>
2026-05-10 17:02:22 +00:00
549 changed files with 10397 additions and 5767 deletions
Loaded 100 of 549 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
mlpy0