mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
e45dc5e9f72ea510bc8aab3248d2bef3be8f2263
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Stefan Walcz
|
b56b903c92 |
feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools) (#12661)
* feat(config): let model aliases carry an opt-in profile block An alias can now act as a profile of its target through an explicit `profile:` block carrying system_prompt, mcp (remote/stdio servers, plus the new mcp.auto) and the new client_tools. ResolveAlias applies only that block on top of a copy of the target, while Name, Model, Backend and the persisted revision stay the target's. The loader key is therefore unchanged and every profile runs on the target's backend process; today a persona needs a separate model config, which loads the weights a second time. The block is opt-in so existing aliases never change: top-level fields on an alias stay ignored, and an alias without a profile block resolves to exactly the target. A profile block on a non-alias config is rejected. A profile with its own MCP servers caches its sessions under the alias name (MCPSessionKey), because the cache slot keyed on the target's name is invalidated whenever its config hash changes and two server sets would tear each other's sessions down. mcp.auto enables all configured servers on /v1/chat/completions when the request does not name any in metadata.mcp_servers, which plain OpenAI clients cannot send. client_tools: drop discards client tools and tool_choice before the request is merged (chat completions, responses, Anthropic messages), so only the model's own MCP tools are offered. Both keys work on any model and inside an alias's profile block. Refs #12548 Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(openai): only auto-enable model MCP when MCP is enabled and permitted mcp.auto turns a plain /v1/chat/completions request into a server-side MCP request. It now applies the same controls as the dedicated MCP routes before any server is selected: the instance switch (LOCALAI_DISABLE_MCP) and, with auth on, the caller's "mcp" feature, checked with auth.HasFeatureAccess against the auth database that the chat route now passes to ChatEndpoint. A denied request is answered as a plain chat, without contacting a server or adding tools, rather than rejected: the client did not ask for MCP, it only reached a model that offers it. Endpoint specs drive ChatEndpoint against a counting MCP fixture server: MCP disabled and a user with chat but without mcp see no tools and no connection; MCP enabled and a user with mcp get the fixture tool. The user specs need the auth build tag (SQLite auth database). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(mcp): resolve profile aliases in the MCP discovery endpoints GET /v1/mcp/servers/:model, /v1/mcp/prompts/:model and /v1/mcp/resources/:model looked the model up with GetModelConfig, which returns the alias stub. A profile alias carries its servers in its profile block, so the stub has none of its own and the endpoints reported an empty list for an alias whose chat requests do use tools. Resolve the alias the way the request middleware does before reading the MCP config, and open the sessions under MCPSessionKey so discovery shares the profile's session cache slot instead of tearing down the target's. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * feat(config): opt-in evidence check for profile aliases A profile with server-side MCP tools (profile.mcp, mcp.auto) runs its tools on the server and returns only the answer text. On the next turn the client sends the history back without those tool calls and results, and the model then tends to skip the tool and answer from nothing. In a measurement on a follow-up question, 0 of 12 turns called the tool and the answers carried invented numbers (e.g. "0" where the true value was 9); with the tool trail kept in the history the tool was called 12 of 12 times. A prompt instruction does not reliably prevent this, so the check sits at the output. New profile keys: profile: evidence_check: numbers: true # enables the check note: "..." # optional, defaults to an English note Rule, per request and stateless: when the check is on, no MCP tool call succeeded on the server during this request (a call that returned an error delivers no evidence and does not count), and the final answer contains a number (\b\d+([.,]\d+)?\b) that occurs in none of the request's input (the profile's system prompt and every message as received, before compression), the note is appended after a blank line. "3,5" and "3.5" count as the same number. In a streamed response the note is one extra content delta before the finish chunk. Choices that hand tool calls to the client are not checked. The setting is only read from a profile block, so plain models and aliases behave as before; a note without numbers: true is rejected at validation. Tests (Ginkgo): the number rule (invented number, number from the input, no numbers, comma/dot decimals, system prompt as evidence, tool ran), the overlay into the resolved config with plain alias, target and a top-level key as counter-checks, validation, and ChatEndpoint specs on a stubbed backend for both paths: note on an invented number, none for a number from the request, none for a profile without the check, none after a successful MCP tool call against an in-process MCP server, and the note again after a failed tool call (arguments that are not JSON). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
Ettore Di Giacinto
|
dcc7bd8548 |
docs: document model failover chains
Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]andEttore Di Giacinto
|
80e3240f2d |
feat(distributed): key scheduling rules by a model alias (#11771)
Node placement and replica rules could only name a model, so an operator who pinned "llama3" to the GPU tier had to rewrite the rule whenever a different model took over that job. An alias already gives a stable name for whichever model serves it, and a rule on that name makes it a deployment slot: repoint the alias and the placement follows. A rule keeps the name the operator chose. Reads resolve that name through the config loader to the model the rule governs, so the reconciler counts, schedules and trims replicas of the target, and the router finds an alias-keyed rule from the target it is already routing. An alias that resolves to nothing governs nothing loadable, so the reconciler skips it and the write paths refuse it. A replica is shared by every name that resolves to it, so only one rule can decide where it runs. The REST and MCP write paths reject a rule whose target another rule already governs. A pair that arrives some other way, such as a seed file or an alias repointed onto a model that already has a rule, resolves in favour of the rule named after the model itself and then the oldest, and the rest are listed as shadowed. The eviction guard is the exception: it matches rules to replicas in raw SQL inside a locking transaction and cannot resolve an alias. It reads a stored target that the reconciler refreshes each tick, and falls back to the rule's own name when that target is empty. Assisted-by: Claude:claude-opus-5 golangci-lint eslint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]andEttore Di Giacinto
|
40d35c0385 |
docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)
* docs: fix CPU image tag (latest, not latest-cpu) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: use canonical localai/localai registry in models guide Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: replace dead llama-stable backend with llama-cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct mitm-proxy intercept config and redaction tier Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fix text-to-audio endpoint and broken notice block Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fix VAD example, stale FAQ, broken link, CLI list, whats-new dump Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: render advanced/reference section indexes (consolidate _index) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove duplicate getting-started build/kubernetes pages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fold container image reference into installation/containers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove stale advanced fine-tuning page (superseded by features/fine-tuning) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fold distribution/longcat/sound pages into their parents Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: make getting-started index accurate and complete Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: carry one concrete model through the getting-started path Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add end-to-end 'build your first agent' walkthrough Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add runtime errors reference; consolidate troubleshooting from FAQ Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add agent actions catalog Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: agent-scoped MCP, skills walkthrough, agentic disambiguation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add concrete gallery install lines to media feature pages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: merge installation into getting-started (URLs preserved via aliases) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add Operations section; move operator pages and P2P API reference Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: journey-ordered top nav and grouped feature sections Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add docs-with-code process gate (PR template + agent instructions) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove em/en dashes from documentation prose Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
LocalAI [bot]andEttore Di Giacinto
|
9565db5f94 |
feat(models): model aliases - redirect a model name to another configured model (#10414)
* feat(config): add model alias field and self-validation Add ModelConfig.Alias (yaml: alias), IsAlias(), and an alias short-circuit at the top of Validate() that rejects self-reference and forbids setting backend/parameters.model on a pure-redirect alias. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): resolve and validate model alias targets in the loader Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(middleware): resolve model aliases and stamp requested/served identity Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(modeladmin): reject alias configs with invalid targets on create/edit Validate alias targets at create/swap entry points (ImportModelEndpoint, EditYAML, PatchConfig) so a dangling, chained, or disabled alias target is rejected at save time rather than surfacing as a runtime error. Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET /api/aliases to list model aliases Adds an admin-gated read-only endpoint that lists every model alias config as {name, target} pairs, backed by the loader's existing GetAllModelsConfigs(). Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(mcp): add set_alias and list_aliases tools Expose model-alias management over the LocalAI Assistant MCP surface: list_aliases (read-only, GET /api/aliases) and set_alias (mutating). SetAlias is swap-first: PATCH /api/models/config-json/:name swaps an existing alias's target (validated, non-destructive) and a 404 falls back to POST /models/import to create a fresh {name, alias} config. The inproc client mirrors this via ConfigService.PatchConfig + a create path modeled on ImportModelEndpoint. Deletion reuses delete_model. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(mcp): replace em dashes in alias tool comments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config-meta): expose alias as a model-select field Add an 'alias' section to DefaultSections() and an 'alias' field override in DefaultRegistry() so the schema-driven React editor renders the new top-level ModelConfig.Alias field as a model picker in its own section. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add alias template card and Manage alias badge Add an 'Alias / Routing' template to the create-flow gallery that seeds a minimal name + alias config, and a read-only 'alias -> target' badge on the Manage Models tab. The capabilities row payload does not carry the alias field, so the badge resolves targets from GET /api/aliases looked up by name. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document model aliases Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(swagger): regenerate for GET /api/aliases Adds the /api/aliases path and AliasInfo schema generated from the ListAliasesEndpoint annotation. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(localai): check os.RemoveAll error in aliases_test Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: correct alias conversion docs and advertise /api/aliases in instructions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(mcp): write alias config 0600 to satisfy gosec G306 The inproc createAlias path wrote the alias YAML with 0644, which gosec flags as a new G306 finding on the PR. The LocalAI process is the sole reader/writer of model configs, so 0600 is correct and keeps the scan clean. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |