mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 00:59:28 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs(addons): describe backends in the library

Browse Source 
Document the entry kinds, how a reader skips a kind it does not know,
the backend entry format, the signer identity pattern, the install
route, the size limit of each kind, the pilot library and the command
that renders it. The backend page links to it.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto committed 2026-10-21 20:15:10 +00:00
1 parent 49b2b151b7
commit 57816ed7e6
2 files changed
+157 -3
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/content/features/addons.md
+148 -3
View File
@@ -76,6 +76,8 @@ loopback carrier with keys that are made at each start and never written to disk
| `--addons-enabled` | `LOCALAI_ADDONS_ENABLED` | `true` | Runs the addon owner. `--addons-enabled=false` turns it off completely: no owner starts, no addon runs, and every addon request is refused as unavailable. |
| `--addons-ownership-mode` | `LOCALAI_ADDONS_OWNERSHIP_MODE` | `local-owner` when authentication is off, `named-user` when it is on | `named-user` gives each account its own access. `local-owner` makes every caller share one owner. A value you set always wins over the default. Any other value stops startup with an error. |
| `--addons-official-library` | `LOCALAI_ADDONS_OFFICIAL_LIBRARY` | `true` | Registers the [official library](#the-official-library) on the first start. `--addons-official-library=false` does not register it. |
| `--addons-backend-library` | `LOCALAI_ADDONS_BACKEND_LIBRARY` | `false` | Also registers the pilot [backend library](#backends-in-the-library) on the first start. |
| `--addons-backend-max-bytes` | `LOCALAI_ADDONS_BACKEND_MAX_BYTES` | `8589934592` (8 GiB) | Largest compressed size of a backend image installed from a library. Add-ons keep their own limit of 512 MiB. |
`named-user` needs authentication. With authentication on and no mode set, LocalAI
uses `named-user`. With authentication off there are no accounts, so LocalAI uses
@@ -108,6 +110,147 @@ source routes), it stays removed, and a `sources.json` you already have is never
changed. To start without it, set `--addons-official-library=false` before the first
start. To get it back, add a source with the URL above.
## Backends in the library
A library lists more than add-ons. Every entry has one schema and an optional `kind`
field. The kinds are:
| Kind | What the entry installs | State |
|---|---|---|
| `addon` | A supervised add-on package. It is the default: an entry without `kind` is an add-on, so every index written before kinds existed stays valid. | Supported |
| `backend` | A backend image for the backends directory. | Supported (phase 1, pilot) |
| `model` | A model. | Reserved. No reader installs it yet. |
The backend gallery (`backend/index.yaml`) stays the source of truth and its install
path does not change. A library entry of kind `backend` is one more way to see and
install a backend that the gallery already lists.
### How a reader treats an unknown kind
A reader keeps the entries of the kinds it knows and skips every other entry. It does
not read the fields of a skipped entry, so a newer kind can carry fields this version has
never seen. The index is not refused. A skipped entry is reported in two places:
- the refresh result and the state of the library have `skipped` (a count) and
`skippedKinds`. The Libraries tab shows them;
- the refresh writes a warning to the log.
A kind that is not a plain lowercase name, or that is not text, is a malformed entry
and the index is refused. An unknown field on an entry of a known kind is still refused.
Search lists add-ons unless the caller asks for a kind: `catalog-search` takes an optional
`kind` argument (`addon`, `backend` or `all`).
### A backend entry
```yaml
- kind: backend
addonId: cpu-local-store # the backend name, also its directory name
alias: local-store # optional: the name a model uses
name: cpu-local-store
description: Local Store is a local-first vector database.
version: 0.0.0-latest.0a1056fd8633
channel: latest
artifact: oci://quay.io/go-skynet/local-ai-backends:latest-cpu-local-store
digest: sha256:0a1056fd863328adf9d1097808aa2b52f68c4ca2e5385100a754755b3ca6218f
os: linux
arch: amd64
```
The `digest` is the digest of the image for the platform. The install uses the image by
digest and never by tag. A backend entry must use an `oci://` artifact. It has no main
page or icon, and only a backend entry can set `alias`. The backend name is a directory
name: it has no `/` and no `..`.
### Signer identity pattern
The signature policy belongs to the library, as for add-ons. A backend library can name
the signer with `identityRegex` instead of an exact `identity`, so images that the merge
workflow signed on master and on a tag both verify:
```json
{
"requireSignature": true,
"issuer": "https://token.actions.githubusercontent.com",
"identityRegex": "^https://github\\.com/mudler/LocalAI/\\.github/workflows/backend_merge\\.yml@refs/(heads/master|tags/.+)$"
}
```
Set `identity` or `identityRegex`, not both. The pattern must be one expression that
starts with `^` and ends with `$`. A pattern with a top-level alternation, such as
`^a|b$`, is refused, because one branch would match a part of the identity only. An
add-on install refuses a library policy that has a pattern: add-on libraries name the
signer exactly.
### Install a backend from a library
An administrator installs a backend from the Browse tab of the Add-ons page. The entry
shows its kind. The page sends the install to the backend route, not to the add-on
lifecycle: an add-on stays disabled until you enable it, a backend is ready to use when
the install ends.
The route is `POST /backends/library/install`. It is for administrators and exists only
when the add-on owner runs on this instance:
```bash
curl -X POST http://localhost:8080/backends/library/install \
-H 'Content-Type: application/json' \
-d '{"sourceId":"official-backends","name":"cpu-local-store","version":"latest"}'
```
The request names the entry and nothing else. The library resolves the image, the digest
and the policy, so a request cannot change them. The install:
1. resolves the entry of kind `backend` for this platform;
2. refuses it when `--require-backend-integrity` is on and the library does not require a
signature;
3. pins the image by digest, verifies the signature before it fetches any layer and refuses
an image over the size limit (the layers are summed from the manifest);
4. installs with the same code as the gallery, so the backend lands in the same directory
with the same `metadata.json`, plus `library`, the ID of the library it came from.
The install runs inside the request and answers when it ends: `200` on success, `404` for
an unknown entry, `409` when the name is already installed from another origin (a gallery,
another library or a manual install; nothing changes), `413` over the size limit and `400`
when strict mode refuses the library. Installing again from the same library is a
reinstall.
A library install touches only the node that serves the request. In a distributed
deployment the backend is installed on that node only; the per-node routes of the backend
gallery are unchanged.
### Size limits
The limit is on the compressed size of the package or image, by kind:
| Kind | Default | Configure |
|---|---|---|
| `addon` | 512 MiB | No |
| `backend` | 8 GiB | `--addons-backend-max-bytes` |
### The pilot backend library
`gallery/backends/index.yaml` lists a pilot: the CPU build of `local-store` for
`linux/amd64`. Register it on the first start with `--addons-backend-library`, or add it
later in the Libraries tab with this URL and the policy above:
```
https://raw.githubusercontent.com/mudler/LocalAI/master/gallery/backends/index.yaml
```
The entries are rendered from the backend gallery. The command reads the registry, so the
digests are never edited by hand. Render the file again after the backend images are
rebuilt:
```bash
go run ./scripts/backends/cmd/backendcatalog \
--index backend/index.yaml \
--backends cpu-local-store \
--out gallery/backends/index.yaml
```
Phase 1 does not include models, the other backends, distributed install from a library,
or a change to the gallery.
## Manifest
`ParseManifest` accepts a single JSON object, at most 64 KiB, and rejects unknown
@@ -371,7 +514,8 @@ A catalog (the library index) is one YAML document: a block list of releases wit
mainPage: databases
```
Unknown fields, a second YAML document and duplicate keys are rejected. A JSON
An entry may carry a `kind` (see [Backends in the library](#backends-in-the-library));
without it the entry is an add-on. Unknown fields, a second YAML document and duplicate keys are rejected. A JSON
array is refused too, even though YAML can read it: write the index as a block
list. An empty library is `[]`. Package manifests (`manifest.json`) stay JSON; only
the library index is YAML. Source identity and verification policy always
@@ -464,7 +608,8 @@ duplicate paths, file/directory collisions, and a supplied `receipt.json` are
rejected. OCI layers cannot overwrite earlier paths: packages are flattened
application bundles, not general-purpose container root filesystems.
Defaults cap compressed downloads and extracted content at 512 MiB and archive
Defaults cap compressed downloads and extracted content at 512 MiB (a backend image has its own
[limit](#size-limits)) and archive
entries at 10,000; owner configuration may set smaller positive limits. Staging
is private, cancellation-aware, and removed on failure. Existing packages are
never replaced or deleted. Commit publishes a unique read-only package directory
@@ -1927,7 +2072,7 @@ writes keep the operation identity and outcome-unknown rule of the lifecycle act
| Action | Kind | Arguments | Payload |
|---|---|---|---|
| `catalog-search` | read | `text`, `sourceId`, `offset`, `limit` (at most 100) | `{releases, total}`; only releases for the owner's configured target platform are listed, and `total` counts that set |
| `catalog-search` | read | `text`, `kind` (`addon` by default, `backend` or `all`), `sourceId`, `offset`, `limit` (at most 100) | `{releases, total}`; only releases for the owner's configured target platform are listed, and `total` counts that set |
| `catalog-details` | read | request `addonId`; `sourceId`, `version` | `{release, installed}`; `installed` is `null` or `{version, generation, enabled}` |
| `catalog-sources` | read | none | `{sources}`; each source has `refresh` once the owner fetched it |
| `addons-installed` | read | none | `{addons}`, the installed view below |
docs/content/features/backends.md
+9
View File
@@ -209,6 +209,15 @@ the repository, so upgrade every node, workers included, before you rely on it:
}
```
## Installing from a library
A library of the [Add-ons page]({{% relref "features/addons" %}}) can list backends too. The
backend gallery stays the source of truth: a library entry is one more way to see and
install a backend from it, with the same signature check, a pinned image digest and a
size limit (8 GiB by default). The Browse tab of the Add-ons page shows each entry with
its kind. See [Backends in the library]({{% relref "features/addons#backends-in-the-library" %}})
for the entry format, the signer pattern and the install route.
## Pre-installing Backends
You can pre-install backends when starting LocalAI using the `LOCALAI_EXTERNAL_BACKENDS` environment variable: