mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
6a80e23733
Add a routing middleware stack and a cloud-proxy backend. * cloud-proxy: a Go gRPC backend that forwards OpenAI- and Anthropic-shaped chat requests to upstream providers, with an optional translate mode (OpenAI request -> Anthropic /v1/messages -> OpenAI response) and full tool-calling support. * routing: admission control, content-aware model routing (embedding cache + classifier + rerank + Arch-Router score), PII detection/redaction (regex + NER) with streaming filter and OpenAI/Anthropic adapters, and a per-user/per-key billing recorder backed by GORM or in-memory storage. * middleware: UsageMiddleware records usage via the billing recorder, plus admission, route-model, usage-stamp and trace middlewares. * observability: BackendTrace ring buffer stores full request bodies (capped), MITM proxy emits structured trace events, and router classifier decisions surface at /api/router/decide. * gallery: Arch-Router-1.5B (Q4_K_M and Q8_0). * UI: cloud-proxy model-editor fields, classifier system-prompt and score-normalization config, and a Traces page rendering request bodies. Assisted-by: claude-code:claude-opus-4-7 [Read] [Edit] [Bash] Signed-off-by: Richard Palethorpe <io@richiejp.com>
33 lines
766 B
Plaintext
33 lines
766 B
Plaintext
.idea
|
|
.github
|
|
.vscode
|
|
.devcontainer
|
|
models
|
|
backends
|
|
volumes
|
|
examples/chatbot-ui/models
|
|
backend/go/image/stablediffusion-ggml/build/
|
|
backend/go/*/build
|
|
backend/go/*/.cache
|
|
backend/go/*/sources
|
|
backend/go/*/package
|
|
examples/rwkv/models
|
|
examples/**/models
|
|
Dockerfile*
|
|
__pycache__
|
|
|
|
# SonarQube
|
|
.scannerwork
|
|
|
|
# backend virtual environments
|
|
**/venv
|
|
backend/python/**/source
|
|
|
|
# In-place llama.cpp clone + per-variant build copies. The Makefile
|
|
# clones llama.cpp itself at the pinned LLAMA_VERSION; if a stale
|
|
# local checkout is COPY'd into the image, the `llama.cpp:` target
|
|
# sees the directory and skips re-cloning, so grpc-server.cpp ends
|
|
# up compiled against whatever (likely older) commit the host had.
|
|
backend/cpp/llama-cpp/llama.cpp
|
|
backend/cpp/llama-cpp-*-build
|