diff --git a/.agents/api-endpoints-and-auth.md b/.agents/api-endpoints-and-auth.md index 4ed161725..3ff8b9f68 100644 --- a/.agents/api-endpoints-and-auth.md +++ b/.agents/api-endpoints-and-auth.md @@ -41,7 +41,7 @@ Add routes in the appropriate file under `core/http/routes/`. The file you use d |------|----------| | `routes/openai.go` | OpenAI-compatible API endpoints (`/v1/...`) | | `routes/localai.go` | LocalAI-specific endpoints (`/api/...`, `/models/...`, `/backends/...`) | -| `routes/agents.go` | Agent pool endpoints (`/api/agents/...`) | +| `routes/studio.go` | Chats endpoints (`/api/chats/...`, including `/api/chats/admin/...` behind the admin middleware) and the agent tasks and jobs (`/api/agent/...`, feature `mcp_jobs`) | | `routes/auth.go` | Auth endpoints (`/api/auth/...`) | | `routes/ui_api.go` | UI backend API endpoints | @@ -187,10 +187,10 @@ func MyHandler(c echo.Context) error { Middleware can be composed at different levels. Here are the patterns used in the codebase: -### Group-level middleware (agents pattern) +### Group-level middleware (collections pattern) ```go // All routes in the group share the middleware -g := e.Group("/api/agents", poolReadyMw, agentsMw) +g := e.Group("/api/collections", readyMw, collectionsMw) g.GET("", listHandler) g.POST("", createHandler) ``` diff --git a/.agents/building-and-testing.md b/.agents/building-and-testing.md index 021d555ec..f3915fe33 100644 --- a/.agents/building-and-testing.md +++ b/.agents/building-and-testing.md @@ -45,3 +45,13 @@ Rules (both gates): - **Don't weaken the gate:** never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up. - If a change drops coverage, **add tests** (sort `coverage-summary.json` by line% ascending to find untested code) rather than editing the baseline. When coverage legitimately rises, commit the regenerated baseline (`make test-coverage-baseline` / `test-ui-coverage-baseline`). - The Go gate is **strict — no tolerance**; `covermode=atomic` keeps it deterministic. The UI gate keeps a small tolerance only because its e2e coverage isn't. + +## Other CI gates + +- **Add-on Go tests:** `TEST_PATHS` and `COVERAGE_ROOTS` include `./addons/...` and `./tests/addons/actione2e` on Linux (the Sandbox add-on does not build on macOS). The other `tests/addons/*` suites need fixtures that `scripts/addons/verify.sh` prepares and run in `.github/workflows/addons-tests.yml`. See `docs/content/development/addon-testing.md`. +- **UI unit checks (`make test-ui-unit`, `tests-ui-unit.yml`):** eslint (errors only), the inline-style ratchet, the host UI unit tests, the unit tests of every `addons/*/ui` package and the locale ratchet. +- **Locale ratchet (`make test-i18n`):** every locale must have every namespace and key of English. `core/http/react-ui/i18n-baseline.json` records the gaps that exist today as a count per locale and namespace; the count may only fall. A namespace file that a locale lacks fails the gate whatever the baseline says (the browser would get a 404 for it on every page load). After you add keys to a locale, run `node scripts/i18n-check.mjs --write` in `core/http/react-ui` and commit the lower baseline; `--report` lists every missing key. +- **PostgreSQL stores:** `test.yml` starts a PostgreSQL service and sets `LOCALAI_TEST_POSTGRES_URL`; `make test-postgres` fails when any PostgreSQL store test skips. +- **Shipped bundle:** specs tagged `@production-chunks` (stale chunk recovery, `e2e/production-bundle.spec.js`) run on the production code-split build through `make test-ui-stale-chunk`; the coverage build inlines dynamic imports and skips them. +- **Distributed:** `tests-e2e-distributed.yml` runs `make test-e2e-distributed` (PostgreSQL and NATS in containers) and `make test-distributed-fakebus` (the in-process bus, race detector) nightly and on changes to the distributed code. +- **Auth modes:** `core/http/route_authmodes_test.go` runs every `/api/chats` and `/api/addons` route as the local owner, with a shared API key, and with accounts (session, per-user API key, revoked feature, non-administrator). diff --git a/.agents/distributed-seams.md b/.agents/distributed-seams.md index 2b313ed1f..ba0ee8e6d 100644 --- a/.agents/distributed-seams.md +++ b/.agents/distributed-seams.md @@ -10,11 +10,12 @@ connections to a worker. Only the carrier files import `nats.go` | Seam | Interface | Lives in | Today | |---|---|---|---| | Fan-out | `messaging.Broadcaster` | `core/services/messaging` | NATS client | -| Queues | `messaging.WorkQueue` (producer), `messaging.WorkConsumer` (worker), keyed by `messaging.WorkKind` | `core/services/messaging` (`workqueue.go`) | `NewNATSWorkQueue`, `NewNATSWorkConsumer` | | Control verbs, frontend half | `nodes.NodeCommandSender`, `nodes.FileStager` | `core/services/nodes` | `RemoteUnloaderAdapter`, `S3NATSFileStager` over NATS request/reply; `HTTPFileStager` over HTTP | | Control verbs, worker half | unexported `controlServer` (`handle`, `handleWithProgress`) | `core/services/worker` | `natsControlServer` | | Agent RPC, frontend half | `AgentControl` (package `mcp`) | `core/http/endpoints/mcp` | `nodes.NATSAgentControl` | | Agent RPC, worker half | unexported `agentRPCServer` | `core/cli/agent_worker.go` | `nodes.NATSAgentRPCServer` | +| Chat start, frontend half | `studio.ChatWorkers` | `core/services/studio` | `nodes.NATSChatStarter` | +| Chat start, worker half | `studio.Service.ServeStart`, served by `nodes.ChatStartHandler` | `core/application/studio_chatworker.go` | `nodes.NATSChatStartServer` | | Dial to a worker | `nodes.BackendClientFactory`, `nodes.ModelProber`, `nodes.WorkerNetDialerFor` | `core/services/nodes` | direct dial | Request and reply payloads of the control verbs live in @@ -23,23 +24,36 @@ carrier share one wire format without importing each other. ## Queues -`WorkKind` names the work (`WorkTask`, `WorkMCPCI`, `WorkAgentRun`). -`natsRoute` in `workqueue_nats.go` is the one place a kind becomes a subject and -a queue group. A nil error from `Enqueue` means the carrier accepted the -payload, not that a consumer exists. The NATS carrier ignores the ctx of -`Enqueue`. +There are none. The agent-run, MCP CI and job queues were removed with the old +agents and jobs; chat runs and triggers use the database and advisory locks +(`.agents/distributed-state.md`). A new queue is a new interface in +`core/services/messaging`, with its own subject root added to +`broadcastRoots` in `subject_rules.go`. -`Consume(ctx, kind, maxInFlight, h)` keeps two concurrency models on purpose. -With `maxInFlight` 1 the handler runs inline on the NATS delivery goroutine and -a panic is not recovered. Any other value spawns a recovered goroutine per -delivery; when bounded, the slot is taken on the delivery goroutine. 0 and -negative values are unbounded. `Unsubscribe` stops delivery, then waits for -running handlers, so a handler must not call it. The agent worker asks for -MCP CI with 1 (`startMCPCIConsumer`) and for agent runs with the dispatcher's -`maxConcurrent` (0 from the CLI); specs pin both. -`WithAgentRunRoute` lets an agent worker move the agent-run subject and group. -An empty subject keeps `agent.execute`; an empty queue is kept and makes a -plain subscription, as an explicitly empty `LOCALAI_AGENT_QUEUE` always did. +The chat start is not a queue: it is a request/reply in a queue group, like the +agent RPC. Nothing is buffered, and the reply says whether the turn started. + +## Chat start + +`studio.ChatWorkers.StartChat` hands the start of an unattended run (trigger, +job, connector message) to one chat worker on `chats.start`, queue group +`chat-workers`. The payloads are `workerctl.ChatStartRequest` and +`ChatStartReply`. Only the deadline of ctx bounds the wait, as for the agent +RPC. Without one the start's budget does: `LOCALAI_CHAT_START_TIMEOUT` +(`nodes.WithChatStartTimeout`, default `nodes.DefaultChatStartTimeout`). The +worker gives the handler the same budget. + +- `ErrNoRoute` (wrapped): no worker listens. `Service.SendBackground` runs the + turn on the frontend. +- A timeout or a transport error: the worker may have started the turn. The + start fails and nothing runs on the frontend, so a message cannot run twice. +- A reply with `Error` set: the worker's refusal. The start fails with + `studio.ErrWorkerRefused`. + +The worker half runs the handler inline on the delivery goroutine. A start +only claims the turn and hands the message to the loop, so it is short; the +turn runs in the background. Events and stop of the turn use the existing +`chats.events.*` and `chats.control.*` subjects. ## Control verbs @@ -150,8 +164,20 @@ other seams against a real server, also through Docker. `ForgetNode`. - `clientFor` returns no error; a carrier with no dialer for a node needs that path. -- The agent worker still uses NATS directly for its connection and for agent - events (`agents.NewEventBridge`). +- The agent worker still uses NATS directly for its connection. +- A chat worker (`local-ai run` with `LOCALAI_CHAT_WORKER`) connects as a + frontend does, with the frontend's NATS credentials. It registers itself in + the node registry through the database (an approved `agent` node with + `node.role=chat-worker`, see `registerChatWorkerNode`), so the node list + shows it and the scheduler skips it. A drain does not reach it, and it gets + no minted JWT: that needs `chats.start` (sub, queue) and the + `chats.events.*` / `chats.control.*` subjects in `pkg/natsauth`. +- A chat worker refuses a start that would take it past + `LOCALAI_CHAT_WORKER_MAX_TURNS` running turns (`studio.ErrWorkerFull`), and + the run fails. A full worker cannot pass a start on to another one: NATS + picks the worker of the queue group, not the frontend. +- A start that carries files runs on the frontend: the uploads are not sent + over the bus. - `ReplicaReconcilerOptions` has no `ClientFactory` field. Without a `Prober`, `NewReplicaReconciler` builds its own `tokenClientFactory`, so a carrier that dials through another factory must add the field. @@ -163,16 +189,5 @@ other seams against a real server, also through Docker. - `natsControlServer.subject` refuses a verb it has no subject for, so registration fails. A verb that only one carrier serves needs a per-carrier opt-out where the verbs are registered. -- `WorkHandler` returns only an error. A carrier whose stream handler must send - a terminal reply derives it from the `jobs..result` event the handler - publishes on `events` (`handleMCPCIJob` does this). -- Not additive: the agent-run consumer (`NATSDispatcher.runDelivery`) ignores - the per-delivery `events` publisher and publishes through the process-wide - `EventBridge`, which is built on the NATS client. A second carrier must - change `NATSDispatcher` and `EventBridge`, for example by binding - `handleJob`'s publishes to `events` through a bridge view that shares the - cancel registry. - `controlHandler`'s `undecodable` return is the recorded exception described under Control verbs. -- Agent cancel has no production sender (`EventBridge.CancelExecution` has no - caller), so it is not part of the agent RPC seam. diff --git a/.agents/distributed-state.md b/.agents/distributed-state.md index 00173edfe..868459d53 100644 --- a/.agents/distributed-state.md +++ b/.agents/distributed-state.md @@ -87,6 +87,13 @@ Examples: - Sticky: the failover prober (`core/application/failover_distributed.go`), via `HeldLock`. +A single runner may hand its work to another process and stay the runner. The +chat hosts (triggers, connectors, jobs) keep their `HeldLock` on a frontend +and, with chat workers on, send each turn to a worker (`studio.SendBackground`, +`.agents/distributed-seams.md`). The turn claim in the database, not the lock, +keeps one turn per chat, so the frontend and the worker use the same claim, +heartbeat and reaper. + ### 3. Stateless per request Nothing to share: the feature re-derives everything it needs from the diff --git a/.agents/localai-assistant-mcp.md b/.agents/localai-assistant-mcp.md index dcf5650bf..abdaea2ef 100644 --- a/.agents/localai-assistant-mcp.md +++ b/.agents/localai-assistant-mcp.md @@ -35,6 +35,7 @@ If you ship a REST endpoint without (2) and (3), conversational admins won't see - `pkg/mcp/localaitools/server_test.go` adds the tool name to `expectedFullCatalog` and `expectedReadOnlyCatalog` (if read-only). - Tool dispatch is added to `TestEachToolDispatchesToClient`. - `pkg/mcp/localaitools/httpapi/client_test.go` covers the new HTTP path. +- [ ] An admin route or setting that deliberately gets no tool goes in `adminRoutesWithoutTool` in `pkg/mcp/localaitools/coverage_test.go`, with the reason. The PR description repeats the reason. ## Adding a new skill recipe (no new tool) diff --git a/.agents/preparing-a-release.md b/.agents/preparing-a-release.md index 9f6f9bcfd..4bc6b98c3 100644 --- a/.agents/preparing-a-release.md +++ b/.agents/preparing-a-release.md @@ -11,6 +11,21 @@ A release is not finished when the tag is pushed. The GitHub release, the blog p A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete. +## Add-ons and the add-on catalog + +The first-party add-ons (Tables, Voice, Face, Fine-tuning, Quantization, Studio, Talk, Office, Sandbox) are separate OCI packages built by `.github/workflows/addons-package.yml`, not part of the LocalAI binary. A release ships them with the tag. A release build of LocalAI reads the add-on index attached to its own release and installs the packages that tag built; it must never fall back to the mutable development builds signed by master. + +What happens, and what to check: + +1. **Tag push (`v`).** The workflow packages every add-on for every platform enabled in `addons//package/platforms.yaml`, pushes `-` to the add-on registry, signs it with the workflow identity of the tag, and the `catalog` job renders `addons-index.yaml` (one pinned entry for each add-on and platform) as the artifact `addons-release-index`. No alias moves. Check that every `publish and sign` leg and the `render the release catalog` job are green, and that the entry count is the number of add-ons times their enabled platforms. +2. **Sandbox guest image.** The `sandbox-image` job builds and signs the guest image; the Sandbox package is stamped with its digest (`SANDBOX_GUEST_IMAGE`). A publishing run fails when the digest is missing. Check the digest in the job summary and that the Sandbox page of a test install shows a guest image. +3. **Release published (stable, not a draft).** The `promote` job moves `-latest`, and the `attach the catalog to the release` job downloads `addons-release-index` from the tag run and uploads it to the release as `addons-index.yaml`. Release builds read that URL: `https://github.com/mudler/LocalAI/releases/download/v/addons-index.yaml`. A release without this asset has no installable add-ons. The artifact is kept for 90 days, so publish the release within that time of the tag, or re-run the tag workflow. +4. **Platforms.** A platform is enabled only after its native check passes on a runner of that platform. `scripts/addons/catalog.sh --check` fails the PR when `gallery/addons/index.yaml` does not list every enabled platform. +5. **Smoke test.** On a clean data directory, start the release binary, open Add-ons, Find add-ons, and install Studio. It must resolve to the `-` tag and verify against the identity of the tag. Repeat on macOS and on linux/arm64 when the release covers them. +6. **Sources.** The release binary and the container image read the model and backend libraries (see the cutover steps in `release.yaml` and the `LOCALAI_*_SOURCE` build arguments in the `Dockerfile`). + +The development entries in `gallery/addons/index.yaml` are regenerated with `sh scripts/addons/catalog.sh --channel development --repository quay.io/go-skynet/local-ai-addons --out gallery/addons/index.yaml`; they do not need a change per release. + ## Clip conventions - MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable. diff --git a/.agents/testing-mcp-apps.md b/.agents/testing-mcp-apps.md index c744f2c99..727236282 100644 --- a/.agents/testing-mcp-apps.md +++ b/.agents/testing-mcp-apps.md @@ -1,117 +1,76 @@ # Testing MCP Apps (Interactive Tool UIs) -MCP Apps is an extension to MCP where tools declare interactive HTML UIs via `_meta.ui.resourceUri`. When the LLM calls such a tool, the UI renders the app in a sandboxed iframe inline in the chat. The app communicates bidirectionally with the host via `postMessage` (JSON-RPC) and can call server tools, send messages, and update model context. +MCP Apps is an extension to MCP where tools declare interactive HTML UIs via `_meta.ui.resourceUri`. When the model calls such a tool, the chat shows the app in a sandboxed iframe under the tool line. The app talks to the host with `postMessage` (JSON-RPC) and can call tools of its own server. Spec: https://modelcontextprotocol.io/extensions/apps/overview -## Quick Start: Run a Test MCP App Server +## How it works in the chats page -The `@modelcontextprotocol/server-basic-react` npm package is a ready-to-use test server that exposes a `get-time` tool with an interactive React clock UI. It requires Node >= 20, so run it in Docker: +The MCP connection is on the **server**, not in the browser. There is no browser MCP client, no `AppBridge` and no CORS proxy in this path. + +1. A turn calls a tool that has `_meta.ui.resourceUri`. The tool wrapper (`core/services/studio/mcpapps.go`, `withApp`) runs the tool, keeps the full MCP result, and sets a card on the call with `session.SetToolCard`. The card holds the app reference: server (`connector:` or the model's server name), tool, resource URI, visibility, input and result. +2. The card is stored with the tool entry of the transcript. `GET /api/chats/{id}/messages` returns it as `app` (with `input` and `result`). The live `tool_end` event carries the reference only (the bus drops the payload) and `seq`, the number of that transcript entry. The entry is stored before the event is sent, in the shared store, so any frontend can serve it. + The page shows the app live: on `tool_end` it reads that one entry with `GET /api/chats/{id}/messages?after=&limit=1` and renders the app in the tool card at once. The live card and the transcript line have the same key (`tool-`, `src/utils/chatLive.js`) in one flat list, so the frame is not remounted when the turn ends and the transcript is read again. If the entry cannot be read, the card shows the result as text and the transcript read at the turn end brings the app. +3. `ChatAppFrame.jsx` reads the page with `GET /api/chats/{id}/mcp-apps/resource?server=&uri=` and renders it in `