From 57816ed7e683d68cd674c843a47897f6e6204898 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 21 Oct 2026 18:15:15 +0000 Subject: [PATCH] docs(addons): describe backends in the library Document the entry kinds, how a reader skips a kind it does not know, the backend entry format, the signer identity pattern, the install route, the size limit of each kind, the pilot library and the command that renders it. The backend page links to it. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto --- docs/content/features/addons.md | 151 +++++++++++++++++++++++++++++- docs/content/features/backends.md | 9 ++ 2 files changed, 157 insertions(+), 3 deletions(-) diff --git a/docs/content/features/addons.md b/docs/content/features/addons.md index fbabe0f99..e7c1ca8c2 100644 --- a/docs/content/features/addons.md +++ b/docs/content/features/addons.md @@ -76,6 +76,8 @@ loopback carrier with keys that are made at each start and never written to disk | `--addons-enabled` | `LOCALAI_ADDONS_ENABLED` | `true` | Runs the addon owner. `--addons-enabled=false` turns it off completely: no owner starts, no addon runs, and every addon request is refused as unavailable. | | `--addons-ownership-mode` | `LOCALAI_ADDONS_OWNERSHIP_MODE` | `local-owner` when authentication is off, `named-user` when it is on | `named-user` gives each account its own access. `local-owner` makes every caller share one owner. A value you set always wins over the default. Any other value stops startup with an error. | | `--addons-official-library` | `LOCALAI_ADDONS_OFFICIAL_LIBRARY` | `true` | Registers the [official library](#the-official-library) on the first start. `--addons-official-library=false` does not register it. | +| `--addons-backend-library` | `LOCALAI_ADDONS_BACKEND_LIBRARY` | `false` | Also registers the pilot [backend library](#backends-in-the-library) on the first start. | +| `--addons-backend-max-bytes` | `LOCALAI_ADDONS_BACKEND_MAX_BYTES` | `8589934592` (8 GiB) | Largest compressed size of a backend image installed from a library. Add-ons keep their own limit of 512 MiB. | `named-user` needs authentication. With authentication on and no mode set, LocalAI uses `named-user`. With authentication off there are no accounts, so LocalAI uses @@ -108,6 +110,147 @@ source routes), it stays removed, and a `sources.json` you already have is never changed. To start without it, set `--addons-official-library=false` before the first start. To get it back, add a source with the URL above. +## Backends in the library + +A library lists more than add-ons. Every entry has one schema and an optional `kind` +field. The kinds are: + +| Kind | What the entry installs | State | +|---|---|---| +| `addon` | A supervised add-on package. It is the default: an entry without `kind` is an add-on, so every index written before kinds existed stays valid. | Supported | +| `backend` | A backend image for the backends directory. | Supported (phase 1, pilot) | +| `model` | A model. | Reserved. No reader installs it yet. | + +The backend gallery (`backend/index.yaml`) stays the source of truth and its install +path does not change. A library entry of kind `backend` is one more way to see and +install a backend that the gallery already lists. + +### How a reader treats an unknown kind + +A reader keeps the entries of the kinds it knows and skips every other entry. It does +not read the fields of a skipped entry, so a newer kind can carry fields this version has +never seen. The index is not refused. A skipped entry is reported in two places: + +- the refresh result and the state of the library have `skipped` (a count) and + `skippedKinds`. The Libraries tab shows them; +- the refresh writes a warning to the log. + +A kind that is not a plain lowercase name, or that is not text, is a malformed entry +and the index is refused. An unknown field on an entry of a known kind is still refused. +Search lists add-ons unless the caller asks for a kind: `catalog-search` takes an optional +`kind` argument (`addon`, `backend` or `all`). + +### A backend entry + +```yaml +- kind: backend + addonId: cpu-local-store # the backend name, also its directory name + alias: local-store # optional: the name a model uses + name: cpu-local-store + description: Local Store is a local-first vector database. + version: 0.0.0-latest.0a1056fd8633 + channel: latest + artifact: oci://quay.io/go-skynet/local-ai-backends:latest-cpu-local-store + digest: sha256:0a1056fd863328adf9d1097808aa2b52f68c4ca2e5385100a754755b3ca6218f + os: linux + arch: amd64 +``` + +The `digest` is the digest of the image for the platform. The install uses the image by +digest and never by tag. A backend entry must use an `oci://` artifact. It has no main +page or icon, and only a backend entry can set `alias`. The backend name is a directory +name: it has no `/` and no `..`. + +### Signer identity pattern + +The signature policy belongs to the library, as for add-ons. A backend library can name +the signer with `identityRegex` instead of an exact `identity`, so images that the merge +workflow signed on master and on a tag both verify: + +```json +{ + "requireSignature": true, + "issuer": "https://token.actions.githubusercontent.com", + "identityRegex": "^https://github\\.com/mudler/LocalAI/\\.github/workflows/backend_merge\\.yml@refs/(heads/master|tags/.+)$" +} +``` + +Set `identity` or `identityRegex`, not both. The pattern must be one expression that +starts with `^` and ends with `$`. A pattern with a top-level alternation, such as +`^a|b$`, is refused, because one branch would match a part of the identity only. An +add-on install refuses a library policy that has a pattern: add-on libraries name the +signer exactly. + +### Install a backend from a library + +An administrator installs a backend from the Browse tab of the Add-ons page. The entry +shows its kind. The page sends the install to the backend route, not to the add-on +lifecycle: an add-on stays disabled until you enable it, a backend is ready to use when +the install ends. + +The route is `POST /backends/library/install`. It is for administrators and exists only +when the add-on owner runs on this instance: + +```bash +curl -X POST http://localhost:8080/backends/library/install \ + -H 'Content-Type: application/json' \ + -d '{"sourceId":"official-backends","name":"cpu-local-store","version":"latest"}' +``` + +The request names the entry and nothing else. The library resolves the image, the digest +and the policy, so a request cannot change them. The install: + +1. resolves the entry of kind `backend` for this platform; +2. refuses it when `--require-backend-integrity` is on and the library does not require a + signature; +3. pins the image by digest, verifies the signature before it fetches any layer and refuses + an image over the size limit (the layers are summed from the manifest); +4. installs with the same code as the gallery, so the backend lands in the same directory + with the same `metadata.json`, plus `library`, the ID of the library it came from. + +The install runs inside the request and answers when it ends: `200` on success, `404` for +an unknown entry, `409` when the name is already installed from another origin (a gallery, +another library or a manual install; nothing changes), `413` over the size limit and `400` +when strict mode refuses the library. Installing again from the same library is a +reinstall. + +A library install touches only the node that serves the request. In a distributed +deployment the backend is installed on that node only; the per-node routes of the backend +gallery are unchanged. + +### Size limits + +The limit is on the compressed size of the package or image, by kind: + +| Kind | Default | Configure | +|---|---|---| +| `addon` | 512 MiB | No | +| `backend` | 8 GiB | `--addons-backend-max-bytes` | + +### The pilot backend library + +`gallery/backends/index.yaml` lists a pilot: the CPU build of `local-store` for +`linux/amd64`. Register it on the first start with `--addons-backend-library`, or add it +later in the Libraries tab with this URL and the policy above: + +``` +https://raw.githubusercontent.com/mudler/LocalAI/master/gallery/backends/index.yaml +``` + +The entries are rendered from the backend gallery. The command reads the registry, so the +digests are never edited by hand. Render the file again after the backend images are +rebuilt: + +```bash +go run ./scripts/backends/cmd/backendcatalog \ + --index backend/index.yaml \ + --backends cpu-local-store \ + --out gallery/backends/index.yaml +``` + +Phase 1 does not include models, the other backends, distributed install from a library, +or a change to the gallery. + ## Manifest `ParseManifest` accepts a single JSON object, at most 64 KiB, and rejects unknown @@ -371,7 +514,8 @@ A catalog (the library index) is one YAML document: a block list of releases wit mainPage: databases ``` -Unknown fields, a second YAML document and duplicate keys are rejected. A JSON +An entry may carry a `kind` (see [Backends in the library](#backends-in-the-library)); +without it the entry is an add-on. Unknown fields, a second YAML document and duplicate keys are rejected. A JSON array is refused too, even though YAML can read it: write the index as a block list. An empty library is `[]`. Package manifests (`manifest.json`) stay JSON; only the library index is YAML. Source identity and verification policy always @@ -464,7 +608,8 @@ duplicate paths, file/directory collisions, and a supplied `receipt.json` are rejected. OCI layers cannot overwrite earlier paths: packages are flattened application bundles, not general-purpose container root filesystems. -Defaults cap compressed downloads and extracted content at 512 MiB and archive +Defaults cap compressed downloads and extracted content at 512 MiB (a backend image has its own +[limit](#size-limits)) and archive entries at 10,000; owner configuration may set smaller positive limits. Staging is private, cancellation-aware, and removed on failure. Existing packages are never replaced or deleted. Commit publishes a unique read-only package directory @@ -1927,7 +2072,7 @@ writes keep the operation identity and outcome-unknown rule of the lifecycle act | Action | Kind | Arguments | Payload | |---|---|---|---| -| `catalog-search` | read | `text`, `sourceId`, `offset`, `limit` (at most 100) | `{releases, total}`; only releases for the owner's configured target platform are listed, and `total` counts that set | +| `catalog-search` | read | `text`, `kind` (`addon` by default, `backend` or `all`), `sourceId`, `offset`, `limit` (at most 100) | `{releases, total}`; only releases for the owner's configured target platform are listed, and `total` counts that set | | `catalog-details` | read | request `addonId`; `sourceId`, `version` | `{release, installed}`; `installed` is `null` or `{version, generation, enabled}` | | `catalog-sources` | read | none | `{sources}`; each source has `refresh` once the owner fetched it | | `addons-installed` | read | none | `{addons}`, the installed view below | diff --git a/docs/content/features/backends.md b/docs/content/features/backends.md index 8817fb8a7..6e331740b 100644 --- a/docs/content/features/backends.md +++ b/docs/content/features/backends.md @@ -209,6 +209,15 @@ the repository, so upgrade every node, workers included, before you rely on it: } ``` +## Installing from a library + +A library of the [Add-ons page]({{% relref "features/addons" %}}) can list backends too. The +backend gallery stays the source of truth: a library entry is one more way to see and +install a backend from it, with the same signature check, a pinned image digest and a +size limit (8 GiB by default). The Browse tab of the Add-ons page shows each entry with +its kind. See [Backends in the library]({{% relref "features/addons#backends-in-the-library" %}}) +for the entry format, the signer pattern and the install route. + ## Pre-installing Backends You can pre-install backends when starting LocalAI using the `LOCALAI_EXTERNAL_BACKENDS` environment variable: