mirror of
https://github.com/mudler/LocalAI.git
synced 2026-11-06 04:23:15 -05:00
docs(cloud-proxy): add OrcaRouter passthrough example
Document OrcaRouter as a first-class cloud-proxy upstream. It is an
OpenAI-compatible gateway, so it plugs into the existing passthrough
mode with provider: openai, mirroring the OpenAI/Anthropic examples.
Assisted-by: Claude:deepseek-v4-flash-0731
Signed-off-by: XiaoHuo888-hue <xiaohuo888hue@users.noreply.github.com>
(cherry picked from commit 329ee0ee89)
(cherry picked from commit 6b2d84294d0198bdaa2f27288993eb7cfad0b0b1)
This commit is contained in:
1 parent
9cc7b63d89
commit
48549f5701
1 file changed
+43
-1
@@ -3,7 +3,7 @@ title = "Cloud passthrough proxy"
|
||||
weight = 28
|
||||
toc = true
|
||||
url = "/features/cloud-proxy/"
|
||||
description = "Forward requests to OpenAI, Anthropic, or any compatible provider"
|
||||
description = "Forward requests to OpenAI, Anthropic, OrcaRouter, or any compatible provider"
|
||||
tags = ["Proxy", "Cloud", "Routing", "Advanced"]
|
||||
categories = ["Features"]
|
||||
+++
|
||||
@@ -144,6 +144,48 @@ pii:
|
||||
Anthropic clients hit `http://localhost:8080/v1/messages` with
|
||||
`"model": "claude-sonnet-proxy"`.
|
||||
|
||||
### OrcaRouter passthrough
|
||||
|
||||
[OrcaRouter](https://www.orcarouter.ai) is a unified OpenAI-compatible gateway
|
||||
that routes to 150+ hosted models (Anthropic, OpenAI, DeepSeek, Gemini, Qwen,
|
||||
…) under a single key. It also runs gateway-level, zero-trust security for AI
|
||||
agents on the same endpoint — screening every prompt/response and governing
|
||||
every tool call on a default-deny basis, with no application code changes.
|
||||
|
||||
OrcaRouter speaks the OpenAI chat-completions wire format, so point a
|
||||
cloud-proxy model at it with `provider: openai`:
|
||||
|
||||
```yaml
|
||||
name: orcarouter-auto
|
||||
backend: cloud-proxy
|
||||
|
||||
proxy:
|
||||
mode: passthrough
|
||||
provider: openai
|
||||
upstream_url: https://api.orcarouter.ai/v1/chat/completions
|
||||
api_key_env: ORCAROUTER_API_KEY
|
||||
upstream_model: orcarouter/auto
|
||||
request_timeout_seconds: 300
|
||||
|
||||
pii:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
Then start LocalAI with the key in the environment:
|
||||
|
||||
```bash
|
||||
export ORCAROUTER_API_KEY=sk-orca-...
|
||||
local-ai run
|
||||
```
|
||||
|
||||
Clients hit `http://localhost:8080/v1/chat/completions` with `"model":
|
||||
"orcarouter-auto"` and LocalAI forwards the request to OrcaRouter.
|
||||
|
||||
`orcarouter/auto` lets OrcaRouter pick the best model for each prompt. To pin
|
||||
a specific model, set `upstream_model` to its OrcaRouter id — for example
|
||||
`deepseek/deepseek-v4-pro` or `anthropic/claude-sonnet-4-6`. Leave
|
||||
`upstream_model` empty to pass the client's `model` field through unchanged.
|
||||
|
||||
### Other OpenAI-compatible providers
|
||||
|
||||
Most third-party providers (Together, Groq, DeepInfra, OpenRouter, …) speak
|
||||
|
||||
Reference in new issue
Block a user