mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
feat(addons): installable add-ons foundation and the Tables add-on (#12702)
* feat(addons): add the add-on runtime, navigation and first-party add-ons Add the add-on runtime to LocalAI: manifests, the host operation broker, admin gating by account role, the add-on UI bundle builder and the SDK for add-on pages. Move the navigation to Chat, Models, Add-ons and Settings, with pinned add-ons in the sidebar and an Add-ons drawer. Ship the first-party add-ons (voice, face, fine-tuning, quantization) as packages for linux/amd64, linux/arm64 and darwin/arm64. Intel macOS (darwin/amd64) is enabled in the platform files. Squashed from the review branch. The tree is unchanged; the pre-squash history is kept on a local backup branch. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gosec): tighten file modes and annotate false positives Write job state, configs and imported models with mode 0600, clean the paths that are read from disk, and mark the route tables, the header name and the retry jitter as reviewed false positives. Assisted-by: Claude Code:claude-sonnet-5-5 gosec Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): keep API keys off recovery mutations and same-site cookies POST /api/addons/dispatch forwarded the recovery and update mutations to the owner for any administrator key, which skipped the browser-session rule of their own routes. Refuse those actions there. Also apply the same-origin rule to the legacy token cookie, not only the session cookie, because the global CSRF check lets same-site requests through. Do not fail the start of LocalAI when an add-on library cannot be refreshed; log it and keep the installed add-ons up. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): refuse a reinstall that would migrate retained data unbacked An upgrade takes a verified backup before the new package migrates the data. A reinstall over retained data has no running generation to stop, so the new package would migrate it without one. Refuse a reinstall whose data format differs from the retained one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): release the manager lock while a release is fetched Submit held the state lock across the catalog lookup and the package download, so the apps list and status reads stalled for the whole install. Serialize Submit and Reconcile with their own lock and drop the state lock around the fetch only. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): kill an add-on only after three missed health probes One slow 100 ms probe killed a busy add-on and dropped the work it was serving. Require three misses in a row. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): move settings under the operate hub The Operate hub already has Settings as a tab, so the sidebar no longer shows a separate Settings entry. The /app/settings route stays, and the hub tab still reaches it. The navigation test now expects Chat, Models and Add-ons as the primary items. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): accept same-host browser requests that carry no fetch metadata Browsers do not send Sec-Fetch-* headers over plain HTTP on an address that is not localhost, so every add-on request from a LAN address was refused as cross-site. Keep refusing a Sec-Fetch-Site value other than same-origin. When the header is absent, accept the request only if its Origin, or failing that its Referer, names the same host and port as the request. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): show the server's reason when an add-on request is refused A 403 or 400 answer showed a generic message and hid the reason in the response body, so a refusal by the browser guard could not be told from a missing permission. Show error.message from the body as the server sent it, and keep the generic text when the body has none. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): run the add-on owner in the normal binary The owner was only enabled when an embedding host set the add-on config, so a normal run answered every add-on request as unavailable. Add --addons-enabled (default true) and --addons-ownership-mode (default named-user, or local-owner; anything else stops startup). The node is its own owner: state lives under the data path, and the carrier uses loopback keys made at each start and kept in memory. --addons-enabled=false turns the owner off completely. Named-user mode needs the account database. With authentication off the owner stays off and a warning says how to use local-owner. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): keep the loopback owner address stable across restarts The single-node loopback transport took a new ephemeral port at each start. The owner stores its address in its identity file and refuses to start when the address differs, so every start after the first failed with "invalid addon owner configuration". Choose the port once, keep it in the add-on state root (loopback-port, mode 0600) and reuse it. If the port is taken at a later start, stop with an error that names the port and --addons-enabled=false, because another port would break the identity check. The keys and certificate stay in memory. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): let add-on pages keep the host modules they were written against A page written for the web UI imports React, the router, i18next and a few shared components. The add-on builder refuses a bundle that holds a host file, so moving such a page into an add-on meant rewriting it. Add a compat mode to the shared builder. It answers those imports from a directory of stand-ins that read the SDK, bundles the dynamic import of the page code into the one output file, and collects the style sheets the page imports. A relative import that is not in the add-on's own source resolves to the file at the same path under the compat directory. Widen the SDK for these pages, without a version change: more host components (spinner, modal, lightbox, model picker) and hooks (models, resources, voice profiles, operations, MCP client), a style on icons, the icon names the pages draw, sdk.navigation.openApp to open a page of the app (only addresses under /app, and the way back as the only state) and sdk.portal. The shell passes the query string to the page. The stand-ins cover the router (the tab in the path, the model in the query), the translator over the host locale files, the core API calls through sdk.core.fetch, and byte copies of the pure helpers, with tests that keep the copies equal to the originals. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): ship Studio and Talk as add-ons instead of built-in pages Studio and Talk were compiled into the web bundle and always listed in the Add-ons drawer. They are add-ons now, like Voice, Face, fine-tuning and quantization: a person installs and removes them, the drawer lists them only while they are installed, and they are packaged the same way. Move the pages, their components, hooks and helpers into addons/studio and addons/talk. Each has a manifest, a UI bundle, a release platform matrix and a catalog template, and is built by the compat mode of the shared builder. The core API stays where it is: the pages call it with the signed-in person's session. The history keeps its browser storage keys. The unit tests of the moved helpers move with them. In the core, drop the two pages, the Create group of the sidebar (Chat joins Home and Models in the old layout), the built-in catalog entries (the catalog is empty) and the talk locale namespace. The old addresses (/app/studio, /app/talk and the generator links) redirect to the add-on pages, with the model as ?model=. The Home /studio command is offered only while the Studio add-on is installed and opens it. The release workflow and its tests now list seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): run the Studio and Talk specs against the add-on bundles The pages are add-ons now, so a spec that draws one installs the real built bundle first: the apps list names it and its module and style sheet are served from the build output, the way the add-on owner serves them. studio-addon.js holds that helper, and the Studio and Talk fixtures call it. Update the specs for the new addresses (/app/addons/studio/studio, the model as ?model=), the session permission the add-on route needs, the legacy redirects, the Home /studio command that appears only while the add-on is installed, and the sidebar without a Create group. Add specs for the add-ons themselves: every tab draws with no script error, the shell titles the page once, the old addresses say so when the add-on is not installed, a button of the page reaches a core page with the way back, and the drawer lists both only after they are installed. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): centre the icon of a built-in tile in its icon box The drawer drew the icon of a built-in tile (Studio's palette was one) in a bare 40px box. Only the placeholder box was laid out as a centred grid with the 20px size, so the real icon sat in the top-left corner at the size of the page text. Give the built-in tile the same classes as the placeholder box, and check in a browser that the icon is centred. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): describe Studio and Talk as add-ons Replace the built-in list (now empty) with the Studio and Talk add-on sections, the new targets of the old links, the Home command, and the seven packaged add-ons. Describe the SDK additions the pages use: components, hooks, icons, openApp and portal. Say on the Studio page that it is installed from the drawer. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): click the shell title in the Studio slash-key spec The add-on shell titles the page, and the Studio overview no longer draws a second title, but the generator tabs keep theirs, so a bare h1 locator matched two elements. Click the first, which is the shell's. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): read the loopback port file through os.Root The persisted port file was read with a path built from the add-on root, which gosec reports as a file inclusion through a variable. Open it through os.Root so the name cannot leave the root, even by a symbolic link, and require a regular file with no group or other access. A looser file now fails closed instead of being trusted. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): expect Settings under the Operate hub, not the sidebar Settings moved out of the sidebar into the Operate hub. The navigation spec still expected a sidebar link. Assert that the sidebar has no Settings entry, and that the Operate hub opens Settings as a tab. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): drop Settings from the 404 places Settings is a tab of the Operate hub and not a destination of its own, so the 404 page does not list it. Assert that it is absent. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): drive the media history and TTS specs through the Studio add-on The image, video, sound and speech generators are pages of the Studio add-on now, and the old addresses redirect to it. Install the add-on with the Studio fixtures, open its pages and use its compose card, so the specs run against the page that exists. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): count the first-run steps, not every list item The recommended models are a list inside the second step, so counting all list items read 3 only while the models were still loading. Count the step elements, which does not depend on the load order. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): fail the node refresh by state, not by call count The spec served the list to the first request and failed the rest. The first request is not always the one from the page, so the page could get the failure first and show no list. Serve the list until the page shows it, then fail the refreshes. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): default to local-owner when auth is off A server without authentication has no accounts, so named-user left the add-on owner off. Resolve the unset mode in one place: local-owner when authentication is off, named-user when it is on. A flag or environment value still wins, and an unknown value is still refused. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): register an official add-on library by default Add gallery/addons/index.yaml, a YAML index of the four first-party add-ons (fine-tuning, quantization, voice, face) in the library format. The entries come from addonpkg catalog over locally built linux/amd64 packages, so the digests pin those images. On the first start, when the owner has no sources.json, register the raw GitHub URL of that index on master as the "official" library and write the file. The file is then the record: a library the admin removes stays removed, and an existing file is never changed. --addons-official-library=false skips the default. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): document the ownership default and the official library Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): list Studio and Talk in the first-party index Add the Studio and Talk entries to gallery/addons/index.yaml next to the four existing ones. They come from addonpkg catalog over locally built linux/amd64 packages, so the digests pin those images. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): expect Studio and Talk in the official index The official index now lists six add-ons. Update the test that decodes it so it accepts Studio and Talk. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): add a real icon to each first-party add-on Each icon is a 256 by 256 PNG: a rounded square in one colour of the UI kit palette with a white outline glyph from the kit sprite. Fine-tuning uses sliders, quantization layers, voice waveform, face user, studio palette, talk phone and tables database. They were rendered with headless Chrome from the sprite paths. Assisted-by: Claude Code:claude-sonnet-5-5 chrome Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): ship a committed icon.png instead of the generated square The shared UI builder and the Tables builder copy ui/icon.png from the add-on source tree to the package unchanged when the file exists. They write the generated placeholder only when it does not. Add a fixture with an icon and a test for both cases. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): list the tables add-on in the first-party index Add the Tables entry to gallery/addons/index.yaml next to the six existing ones. It comes from addonpkg catalog over a locally built linux/amd64 package, so the digest pins that image. The official index test now expects seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): make the Voice add-on one page with tabs The Voice add-on had four pages: recognition, library, create and speech, and the drawer listed two of them. It now has one page, "voice", and one drawer entry. Recognition, Library, Text to speech and From a recording are tabs of that page, drawn with the same tab strip the recognition page already used. The tab is the first path segment (voice/library), as in the Studio add-on, and the form that adds a voice is a view of the Library tab (voice/library/new). The old page names stay declared in the manifest. Each one redirects to its tab and keeps the query string, so saved links still open. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): send the old voice routes to the Voice tabs /app/voice, /app/voice-library and /app/voice-library/new now open the matching tab of the Voice add-on page while the add-on is installed and running, and the core pages otherwise. The moved-page helper compares the first segment of the target with the declared pages, so a target can name a tab. The docs and the navigation spec list the new addresses. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): cover the Voice tabs and the old page redirects The add-on unit tests render the tabs instead of the four pages. The Playwright spec checks the tab strip, the address of each tab, the missing Library tab for a member, and that the old page addresses of the add-on redirect to their tab with the query string kept. The migration table test follows the new targets. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(studio): keep the model menu inside a narrow screen The Home model picker sizes its menu from the viewport, and assumes the picker sits at the page gutter. In the Studio composer card the field is further in, so on a 390px screen the menu of the Sound expansion picker ended 7px past the right edge. Size the menu from the field it opens under instead. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): build the add-on bundles once for the whole run Each worker built the Studio and Talk bundles into the same dist directory, and the build empties that directory first. A worker that started, or restarted after a failure, removed the files another worker was serving, so a Studio spec saw "the style sheet of the add-on could not be loaded". Build once in a global setup before the workers start. A run without the setup still builds per worker. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): register the worker when the add-node spec says so The mocked roster added the worker on the third read of /api/nodes. The shell and the page both read it, and which one asks first depends on load, so the page could take the new worker as part of the roster it opened with and never show it as found. The worker now joins after the page shows it is listening. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): hold the gallery answer until the skeleton is seen The spec delayed the answer by a fixed 800 ms and then looked for the skeleton. The assertion polls at growing intervals, so on a slow machine it could step over that window. The answer now waits until the skeleton has been seen, and the spec then checks that the skeleton goes away. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): give the PDF tag the time the other PDF specs have pdf.js and its worker load on first use. The other specs in this file wait up to ten seconds for the result of that load. The first one waited five, and missed the tag on a busy machine. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): build the Face and Fine-tuning bundles in the global setup The navigation spec serves the built Fine-tuning bundle, but nothing built it, so two specs failed on a fresh checkout when the dist folder was missing. The spec also built the Face bundle in a hook that every worker runs, and a build rewrites the folder the other workers serve. The global setup now builds Studio, Talk, Face and Fine-tuning once, before the workers start. The spec builds the two bundles itself only when it runs without the global setup. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): fix the Home answers in the stale pin spec The spec asserts that the page shows no alert. The Home page raises its own alert when the model list fails or a staging operation has failed, and both answers come from the shared test server. A slow reply or a failed install left there counted as an alert about the pin. The spec now fixes both answers, so the alert count tells only about the pin. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): read the server address in the add-node spec from the page The spec expected the register address to carry port 8089, so it failed on any other port. It now expects the origin the page was opened from. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): give the navigation routes spec its own catalog No page is built in today, so the catalog is empty and the three specs that expect the studio and talk entries failed on the list and on the pin seed. They passed only while Studio and Talk were built in. The spec now sets a catalog of two ungated entries for each example and restores the old one afterwards. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(cli): report a failed write of the MLX hostfile The tunnel callback ignored the error of os.WriteFile and set the hostfile variable even when the file was not written. It now logs the error and leaves the variable alone. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): discard the close error of the held listener errcheck flags the deferred Close of the listener that keeps the port busy. The test has no use for that error, so the defer drops it explicitly. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): move the add-on tool tests to Ginkgo The fine-tuning, quantization and add-on host library tests used t.Errorf, t.Fatal and t.Run, which the forbidigo rule rejects. They now use Describe, It and Gomega, with a suite bootstrap in each package. The cases and the assertions are the same. The tables in the host client and the tool schema tests keep one entry per case. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): move the host operation tests to Ginkgo The hostops tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs under one suite bootstrap. Every check and every per-operation case is kept. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): move the pin store tests to Ginkgo The pin service, file store, sweep and shared store contract tests used the testing package calls that forbidigo rejects. The store contract is now a function that registers its specs for each store, so the file store, the memory-only store and the database store run the same cases. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): move the remaining add-on tests to Ginkgo The built-in catalog, tool arguments, loopback carrier, host operation contract, packaging, application, config, CLI and endpoint tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs. The endpoint tests also drop the unchecked Close errors that errcheck reports when the auth tag is on. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): wait for the targeted Reinstall button before the install click The "generic deletion blocks targeted install at confirmation" spec dispatches a popstate to set the target node, then clicks Reinstall with a script click. The click can run before the targeted page has rendered or while the button is still disabled, so no install request is sent and the poll on the install count times out. It failed 1 run in 10 with two workers and on every retry in CI. The spec now waits for the button to be enabled first. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(addons): refresh the voice package digest in the first-party index The Voice add-on became one page with tabs and got a new icon, so the digest in gallery/addons/index.yaml no longer matched the build. Render the entry again with addonpkg catalog over a locally built linux/amd64 package; the main page is now voice. The other entries are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): wait for the target node's Reinstall button after the popstate The "generic deletion blocks targeted install at confirmation" spec dispatches a popstate to set the target node, then waits for a button named Reinstall and clicks it. Until React renders the targeted page, the page still draws the generic Reinstall button. It is enabled and has the same name, so the wait passed on it, the click started a global install, and no node install request was sent. The poll on the install count timed out with the confirm dialog still open. With the CPU throttled to 8x, the spec failed on 20 of 20 runs, and the button it clicked had the title "Reinstall". The spec now waits for the button titled "Reinstall on <node>". With the throttle it passes 30 of 30 runs. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): give the Studio hand-off spec a budget for the cold bundle load The "a result made through a hand-off records which result it came from" spec opens the Studio add-on straight from an address, twice. Each is a cold load: the page shows "Loading" until the add-on module has run, and the next assertion had the default five seconds. The load takes about two seconds on an idle run and seven to ten under CPU load, so the spec failed on a loaded machine with the hand-off note not yet drawn. With the CPU throttled to 6x and eight busy loops running, it failed on 21 of 30 runs. The first assertion after each load now waits up to 20 seconds, and the spec then passed 30 of 30 under the same load. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): carry the identity page styles in the Face add-on The Face page is drawn with the shared identity views, but its style sheet held only the clip and error rules. The page, card, model list, known-faces table, enrol sheet and distance scale classes had no rules. They looked right only when the host had already loaded the style sheet of its own Faces page, which a visit through the add-on drawer does not do. Opened that way, the cards had no gap, the model list showed bullets, and the install buttons ran into the names. The Face sheet now holds the same identity rules as the core sheet, so the page looks the same whichever way it is opened. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): scope the narrow-screen identity rules of the Voice add-on The rules for narrow screens (single-column pairs, smaller tool padding, compact rank rows) were copied into the Voice style sheet without their media query, so they applied at every width. They now sit in a max-width 720px query, and the reduced-motion rule in its own query, as in the core sheet. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): drop the repeated title from the Voice and Face pages The add-on shell already titles the page, so the recognition pages showed the name twice: "Voice" above "Voices", "Face" above "Faces". Keep the supporting line and the model picker, as the Studio page does, and remove the unused title strings. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): replace the Pinned sidebar section with an Apps entry The sidebar no longer lists pinned add-ons. It shows one Apps entry that opens the add-ons drawer, with a Show all link below it that opens the same drawer. Pins are used from the drawer, where a pinned tile keeps its pin and move buttons. The server pins API is unchanged. Remove the PinnedNav component and its locale title, rename the items.addons label to Apps, and update the e2e specs and the add-ons documentation to match. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): assert the single shell title on the Voice and Face pages The add-on shell now shows the one title, "Voice" or "Face", and the page no longer repeats it. The tests still looked for the h1 "Voices" and "Faces". Assert that the add-on page has exactly one h1 and that it is the shell title. The core pages keep their own "Voices" and "Faces" titles. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): drop the repeated title from the Voice Library and Text to speech pages The add-on shell already titles the page "Voice", so these two tabs showed a second heading below it ("Speech voices", "Speak text"). Keep the supporting line, as the Studio and recognition pages do, and remove the unused title strings. The e2e spec now checks that each tab has exactly one h1, the shell title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): list pinned apps under Apps in the sidebar A pinned app now shows as an indented child of the Apps entry, with its icon and title, in pin order. Show all stays the last child. The pins come from GET /api/addons/pins through the navigation context, and the apps from the add-on registry, which the drawer feeds on every list read, so the links follow the drawer. A pin without an installed app is not drawn, nothing extra shows when no app is pinned, and the children are hidden when the sidebar is collapsed to the rail. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): say that pinned apps are listed under Apps in the sidebar The sidebar lists the pinned apps between Apps and Show all, in pin order. The pins section and the navigation section said that pins do not appear in the sidebar. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): remove the Show all link under Apps in the sidebar The link repeated the Apps entry, which opens the same drawer, and looked redundant when no app was pinned. Apps is now the only add-on entry; pinned apps still list under it in pin order. The link, its locale key and the style comment are gone, and the e2e spec and the add-ons docs now say that Apps is the only entry. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): show a first-run card on Apps when nothing is installed An administrator who opens Apps with no add-on installed now sees a short card with three steps: choose a library, browse and install, start the add-on. Its button opens the Browse tab. The card is hidden once an add-on is installed and for anyone who cannot install. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): hide the pinned apps in the icon rail When the window is narrower than 1024 px the sidebar shrinks to the icon rail by CSS alone, without the collapsed class. The pinned app links were still drawn there, with the indent meant for the full sidebar, so their icons sat to the right of the rail and were cut off. Hide the pinned app links in both rail states. The Apps entry keeps its centred icon. The e2e spec checks that the rail shows no pinned app and that every icon is centred at 1280 px collapsed and at 1000 px. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): stop the backend redirect test racing the installed list The test reads the redirected URL right after the path changes. The Backends page then loads the installed list and drops a selected backend that is not installed, so the backend parameter was sometimes already gone when the test read it. The mock server has no llama-cpp, and about 15 of 40 repeated runs failed on the backend assertion. The redirect itself was correct, including show_all. Serve the selected backend in the installed list, so the page keeps the selection and the assertions stay as strict as before. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): accept a same-host Referer or Origin for human confirmation Browsers send no Sec-Fetch-* headers over plain HTTP on an address that is not localhost, so creating a resource from a LAN address failed with "Creating a resource requires your own browser session". When Sec-Fetch-Site is present it must still be same-origin with an empty destination, as before. When it is absent, Origin or, failing that, Referer must name the request host and port. A destination or mode that is present must still be the expected one. This matches the rule in BrowserProtection. The credential, token cookie, API key and tool-context checks are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep hub entries off the Apps route in the sidebar The Build hub still owned /app/addons through its legacy Add-ons tab, so the sidebar lit both Apps and Build on the Apps page. A tab whose gate is closed now owns no route when the sidebar asks, so only the Apps entry is active there. The hubs keep their own routes, and the legacy navigation still gives /app/addons to the Build hub. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): label the pin control on app tiles and in the detail view The pin was a small icon-only button and hard to find. Each app tile now shows a button reading Pin, or Pinned with a filled pin, and the detail view has the same control in its header. Both toggle the same stored pin. The Apps grid lists pinned apps first, in pin order, then the others. The sidebar list and the server pins API are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): check the Voice From a recording tab has one title The Library and Text to speech tabs already assert that the add-on shell title is the only h1. Do the same for the From a recording tab, so a heading added there later fails the navigation spec. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): stop the add-on shell drawing a title over add-on pages The app decides what its page shows. The shell drew the add-on name as a header above every add-on page, so an add-on could not choose its own title. The shell now draws no header on a live, loading or frozen add-on page; each add-on titles its own pages. The unavailable and missing-page states keep the name, because no add-on page is drawn there. The runtime and shell specs now check that the shell adds no page title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): let each add-on draw its own page title The shell no longer titles add-on pages, so each add-on draws one h1 per page. Voice titles its tabs "Voices" (Recognition and From a recording), "Speech voices" (Library) and "Speak text" (Text to speech); "Add a speech voice" already had its own. Face draws "Faces", Studio "Studio" and Talk "Talk". Tables promotes the database list and database name from h2 to h1. Fine-tuning and Quantization already drew a title. The add-on unit tests and the navigation and Studio specs now assert exactly one h1 per page, with the page's own title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): add a kind to library entries and read unknown kinds tolerantly A library entry now has an optional kind: addon (the default when it is absent), backend or model. Every index written before kinds existed stays valid and its entries are add-ons. A reader keeps the entries of the kinds it knows and skips the others without reading their fields, so a newer kind never makes an older host refuse the whole index. The refresh result and the library state report how many entries were skipped and of which kinds, and the refresh logs a warning. A kind that is not a plain kind name still makes the index malformed. Search lists add-ons unless the caller asks for a kind, and Resolve returns add-ons only, so an add-on install cannot be pointed at a backend image. A library policy may name the signer with an anchored identityRegex instead of an exact identity, so builds from master and from tags both verify. An add-on install refuses a policy that carries one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(backends): install a backend from a library entry A library can now list backends. An entry of kind backend names the image of the backend gallery by digest, and POST /backends/library/install installs it on the local node. The gallery stays the source of truth and its install path is the one used: the library resolves the entry, takes the signature policy from the library, pins the image by digest and calls the same install as the gallery. Installing a name that came from a gallery, another library or by hand is refused and changes nothing. The signature policy of a backend library names the signer with an anchored identityRegex, so images signed by the merge workflow on master and on tags both verify. A pattern that is not anchored is refused. An install is bounded by a size limit per kind: add-ons keep 512 MiB and a backend image defaults to 8 GiB, set with --addons-backend-max-bytes. The limit reads the manifest after the signature check and before any layer is fetched. The metadata of an installed backend records the library it came from. scripts/backends/cmd/backendcatalog renders entries from backend/index.yaml and pins them against the registry. The pilot is the CPU build of local-store, in gallery/backends/index.yaml. The library is registered on the first start only with --addons-backend-library, which is off by default. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): show the kind of library entries and install backends from them The Browse tab lists the entries of every kind this host reads and marks each one as an add-on or a backend. A backend installs through the backend route and not through the add-on lifecycle: the confirmation says that the backend gallery is not changed and that a backend runs native code, and a refusal of the host, such as a name installed from another origin, is shown with its own reason. The details give the kind, the alias and the signer pattern. The library editor can name the signer with a pattern. Without this a save would drop the pattern of a backend library. The Libraries tab reports the entries a refresh skipped because their kind is not read. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): describe backends in the library Document the entry kinds, how a reader skips a kind it does not know, the backend entry format, the signer identity pattern, the install route, the size limit of each kind, the pilot library and the command that renders it. The backend page links to it. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(mcp): install a backend from a library entry The backend library route installs a backend from an add-on library entry, and the admin REST surface and the MCP catalog must stay in step. Add the install_library_backend tool with an in-process and an HTTP client, the route map entry, the safety-rule listing and a skill recipe. Listing library entries stays on the add-on routes, which the assistant does not expose, so no list tool is added. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): carry the add-on library keys in every locale The add-on library and backend library strings existed only in English, so the other locales had no value for them. Add the missing keys of the addons namespace to de, es, id, it, ko, pt-BR and zh-CN. The values are the English text until a translation is contributed. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Revert "feat(mcp): install a backend from a library entry" This reverts commit7493cff727. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Revert "docs(addons): describe backends in the library" This reverts commit57816ed7e6. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Revert "feat(ui): show the kind of library entries and install backends from them" This reverts commit49b2b151b7. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Revert "feat(backends): install a backend from a library entry" This reverts commit1d8d3e3421. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Revert "feat(addons): add a kind to library entries and read unknown kinds tolerantly" This reverts commit484813f856. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): drop the backend library keys from the locales Revert the part of39f77ae29that added strings for backend libraries: the kind labels, the backend install texts, the signer identity pattern and the skipped entries notice. The add-on library keys stay. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add an add-ons control to the chat header Add-on tools were on or off only through the /addons command, with a toast and no state a person could see. The chat header now has an Add-ons button in the style of the MCP dropdown. It lists each running add-on that ships tools with a switch, and a badge shows how many are on. The chat keeps the ids that are on (addonToolIds) next to the older boolean, and both are saved with the chat. A chat that has only the boolean means every add-on. The request sends addon_tools alone when every add-on is on, and adds addon_tool_ids when only some are. /addons turns all on or all off and uses the same state. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): let a chat request name the add-ons whose tools it uses addon_tools=true still offers the tools of every add-on the caller may use. A new addon_tool_ids metadata key, a comma-separated list of add-on ids, narrows that to some add-ons. The list counts only with the opt-in, and a list that names nothing offers no tool rather than all of them. The executor drops the tools of add-ons that are not listed and refuses a call to one as an unknown tool, so a model cannot reach a tool the person turned off. Both keys are consumed so they do not reach the backend. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): cover the per-add-on tool list and the chat header control Go tests cover the list form (the list filters the tools offered and the calls allowed) and the unchanged boolean form. Unit tests cover the chat add-on rules. Playwright specs cover the header control: listing only add-ons that ship tools, toggling each one, the count, /addons in step with the switches, and the request metadata. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): check each add-on page has one title when not enabled The shell names an add-on only where the add-on page cannot mount (not enabled, failed, no UI, bundle not loaded, no such page). Once the page mounts it draws its own title, including when it says it is not enabled on this server, and the shell draws none. The shell code already works this way; this change states the rule beside the unavailable state and pins it in the specs. The shell spec now expects exactly one title in the unavailable states. The navigation spec checks the Face page without a face model (the add-on title only) and the Face page of a stopped add-on (the shell title only). Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): upgrade add-ons on their own when the setting is on A new auto_upgrade_addons setting (flag LOCALAI_AUTO_UPGRADE_ADDONS, default off) lets the owner upgrade an installed add-on when its library lists a newer version. It follows the backend setting: a startup flag, a runtime setting, and the same schedule as the backend update check (a first pass shortly after start, then every six hours). Each pass fetches the library of every installed add-on, and for each available update builds the same pinned upgrade command an administrator sends and runs it through the same lifecycle path. The channel rules, the signer identity, the digest checks, the SDK compatibility check and the backup before the upgrade all apply unchanged. The only difference is the gate: the owner acts on its own setting, so no administrator is asked. A refused or failed upgrade changes nothing and is logged; a candidate that failed is not retried until the library lists another digest. With the setting off, nothing runs and upgrades stay manual. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the Upgrade add-ons automatically toggle to Settings The toggle sits beside Upgrade backends automatically and is off by default. It saves the auto_upgrade_addons runtime setting. The English locale gets the label key, and the Settings test fixture lists the new field. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): cover the automatic upgrade setting and document it The setting defaults to off in the config, the runtime settings and the flag, and is read back from the saved settings file. With it on, a real owner upgrades an installed add-on through the manager; with it off, nothing changes, and a change of the setting at run time takes effect on the next pass. The manager specs show the upgrade goes through the same pinned path as the manual one, and that a corrupted package, a failed signature policy, an incompatible release, a missing update checker and a caller who is not an administrator each leave the installed version in place. The new specs are added to the lifecycle case inventory. The add-ons page describes the setting, its schedule and its default. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(addons): name the library entry identifier id The library entry of an add-on used addonId while the shared envelope and the model entries use id. Release now has the field ID, written as id, and every writer emits it: the index encoder, the package workflow templates, the catalog generator and gallery/addons/index.yaml. The digests in the index did not change. DecodeIndex still reads addonId as a legacy alias for one release, so an existing library and a library served by an older tool keep working. An entry that holds both names is refused. The Apps UI reads id and falls back to addonId. Assisted-by: Claude Code:claude-sonnet-5-5 go node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): use the id field in the auto-upgrade test release The auto-upgrade test arrived after the identifier rename and still named the release field AddonID, so the application tests did not build. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): annotate the auto-upgrade actor label for gosec The actor label looks like a credential to gosec G101. It is a fixed name for upgrades the owner starts itself. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(schema): persist the per-chat add-on tool fields The chat header persists addonTools and addonToolIds, but Conversation had no fields for them, so the server dropped them on save. Add both and update the generated swagger files by hand because swag is not installed here. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): count the add-ons auto-upgrade setting in the settings groups The setting sits in the Backends and galleries group, once. The page now lists 54 settings plus the three branding images, so the spec expects 57. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): pause the fake clock ahead of now in the clear-all spec The fake clock runs from install, so pausing at the current time failed with 'Cannot fast-forward to the past' on a slow runner. The save itself is synchronous and was not lost. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): hold the Studio capabilities reply longer in the skeleton spec 800 ms was shorter than a slow runner needs to render the page, so the skeleton was gone before the check. The product shows it. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(i18n): add the French add-ons locale Master added the fr locale after this branch forked. The add-ons unit test requires every locale to carry the add-ons namespace, so fr needs its own addons.json. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler-agentandEttore Di Giacinto authored and GitHub committed
1 parent
099d6c3dd0
commit
e45dc5e9f7
964 files changed
+159560
-515
No files matched your search
@@ -0,0 +1,85 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
//go:build auth
|
||||
|
||||
package application
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
"github.com/mudler/LocalAI/core/schema"
|
||||
"github.com/mudler/LocalAI/core/services/addons/hostops"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
type recordingExporter struct {
|
||||
calls int
|
||||
}
|
||||
|
||||
func (r *recordingExporter) Checkpoints(context.Context, string, string) ([]hostops.Checkpoint, error) {
|
||||
return []hostops.Checkpoint{{Step: 1, Path: "/out/ckpt-1"}}, nil
|
||||
}
|
||||
|
||||
func (r *recordingExporter) ExportModel(context.Context, string, string, schema.ExportRequest) (string, error) {
|
||||
r.calls++
|
||||
return "/models/m", nil
|
||||
}
|
||||
|
||||
// With an account database the role decides in every mode, so the local-owner
|
||||
// mode does not make a non-administrator an administrator.
|
||||
var _ = Describe("the admin check with an account database", func() {
|
||||
It("follows the role in every mode", func() {
|
||||
db, err := auth.InitDB(filepath.Join(GinkgoT().TempDir(), "auth.db"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
mk := func(id, role string) {
|
||||
Expect(db.Create(&auth.User{ID: id, Email: id + "@example.com", Role: role, Status: auth.StatusActive}).Error).To(Succeed())
|
||||
}
|
||||
mk("admin", auth.RoleAdmin)
|
||||
mk("granted", auth.RoleUser)
|
||||
Expect(auth.UpdateUserPermissions(db, "granted", auth.PermissionMap{"fine_tuning": true})).To(Succeed())
|
||||
|
||||
check := addonAdminCheck(db)
|
||||
Expect(check(contract.Actor{UserID: "granted", Mode: "local-owner"})).To(BeFalse(),
|
||||
"a non-administrator is an administrator in local-owner mode with an account database")
|
||||
Expect(check(contract.Actor{UserID: "admin", Mode: "local-owner"})).To(BeTrue(),
|
||||
"an administrator is refused in local-owner mode")
|
||||
|
||||
in := hostops.NewInFlight()
|
||||
b := hostops.New(in, addonFeatureCheck(db), contract.DefaultLimits())
|
||||
b.SetAdminCheck(check)
|
||||
ex := &recordingExporter{}
|
||||
Expect(hostops.RegisterEngineModels(b, ex, nil)).To(Succeed())
|
||||
|
||||
serve := func(user string) (int, contract.HostResponse) {
|
||||
end := in.Begin(contract.Envelope{
|
||||
Request: contract.Request{AddonID: "fine-tuning", OperationID: "op1", Generation: 1, Action: "start", Arguments: json.RawMessage(`{}`)},
|
||||
Authority: contract.Authority{Actor: contract.Actor{UserID: user, CredentialID: "c1", Mode: "local-owner"}, Scope: "apply", ExpiresAt: time.Now().Add(time.Minute)},
|
||||
})
|
||||
defer end()
|
||||
body, _ := json.Marshal(contract.HostRequest{OperationID: "op1", Op: "finetune.export", Arguments: json.RawMessage(`{"jobId":"j1"}`)})
|
||||
req := httptest.NewRequest(http.MethodPost, "/host", bytes.NewReader(body))
|
||||
req.Header.Set(contract.HostCredentialHeader, "secret")
|
||||
rec := httptest.NewRecorder()
|
||||
b.Handler("fine-tuning", "secret", []string{"finetune."}).ServeHTTP(rec, req)
|
||||
var out contract.HostResponse
|
||||
_ = json.Unmarshal(rec.Body.Bytes(), &out)
|
||||
return rec.Code, out
|
||||
}
|
||||
|
||||
code, out := serve("granted")
|
||||
Expect(code).To(Equal(http.StatusForbidden))
|
||||
Expect(out.Error).ToNot(BeNil())
|
||||
Expect(out.Error.Code).To(Equal(contract.HostFeature))
|
||||
Expect(ex.calls).To(BeZero(), "service called for a non-administrator")
|
||||
code, out = serve("admin")
|
||||
Expect(code).To(Equal(http.StatusOK), "administrator: %+v", out.Error)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,20 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// Without an account database the local owner is the administrator, and no
|
||||
// other actor is.
|
||||
var _ = Describe("the admin check without an account database", func() {
|
||||
It("trusts only the local owner", func() {
|
||||
check := addonAdminCheck(nil)
|
||||
Expect(check(contract.Actor{UserID: "local-owner", Mode: "local-owner"})).To(BeTrue(),
|
||||
"the local owner is not the administrator without an account database")
|
||||
Expect(check(contract.Actor{UserID: "u1", Mode: "named-user"})).To(BeFalse(),
|
||||
"a named user is an administrator without an account database")
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,49 @@
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/xlog"
|
||||
)
|
||||
|
||||
// runAddonAutoUpgrade runs the add-on upgrade pass on the schedule of the
|
||||
// backend update check: once after a short delay, then on every interval. The
|
||||
// setting is read at each pass, so a change in the Settings page takes effect
|
||||
// without a restart.
|
||||
func (a *Application) runAddonAutoUpgrade(ctx context.Context, initialDelay, interval time.Duration) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(initialDelay):
|
||||
}
|
||||
a.autoUpgradeAddons(ctx)
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
a.autoUpgradeAddons(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// autoUpgradeAddons is one pass. With the setting off it does nothing at all:
|
||||
// no library fetch and no upgrade. With it on, the owner runs the same pinned
|
||||
// upgrade an administrator starts by hand.
|
||||
func (a *Application) autoUpgradeAddons(ctx context.Context) {
|
||||
m := a.addonManager
|
||||
if m == nil || !a.applicationConfig.AutoUpgradeAddons {
|
||||
return
|
||||
}
|
||||
results := m.AutoUpgrade(ctx)
|
||||
upgraded := 0
|
||||
for _, r := range results {
|
||||
if r.Upgraded {
|
||||
upgraded++
|
||||
}
|
||||
}
|
||||
xlog.Debug("Add-on auto-upgrade pass finished", "candidates", len(results), "upgraded", upgraded)
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/core/services/addons"
|
||||
"github.com/mudler/LocalAI/core/services/addons/catalog"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("add-on automatic upgrade setting", func() {
|
||||
// start boots a real owner against a library with one add-on at 1.2.3,
|
||||
// installs it, then lists 1.2.4. It returns the application and a function
|
||||
// that reports the installed version.
|
||||
start := func(autoUpgrade bool) (*Application, func() string) {
|
||||
requireAddonRuntime()
|
||||
c := config.AddonConfig{Enabled: true, InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", StorageID: "local",
|
||||
Root: filepath.Join(realTempDir(), "addons"), AdminCheck: func(context.Context, contract.Actor) error { return nil },
|
||||
Host: contract.HostCompatibility{Host: 1, API: 1, Protocol: 1, SDK: 1, DataFormat: 1, OS: "linux", Arch: "amd64"}}
|
||||
DeferCleanup(func() {
|
||||
_ = filepath.Walk(c.Root, func(p string, i os.FileInfo, e error) error {
|
||||
if e == nil && i.IsDir() {
|
||||
_ = os.Chmod(p, 0700)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
})
|
||||
manifest, e := os.ReadFile("../../tests/addons/fixtures/packages/manifest.json")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
var mu sync.Mutex
|
||||
archives := map[string][]byte{}
|
||||
var releases []catalog.Release
|
||||
var url string
|
||||
publish := func(version string) {
|
||||
var m map[string]any
|
||||
Expect(json.Unmarshal(manifest, &m)).To(Succeed())
|
||||
m["version"] = version
|
||||
data, _ := json.Marshal(m)
|
||||
var b bytes.Buffer
|
||||
tw := tar.NewWriter(&b)
|
||||
for name, body := range map[string][]byte{"manifest.json": data, "bin/sample": []byte("binary " + version)} {
|
||||
Expect(tw.WriteHeader(&tar.Header{Name: name, Mode: 0755, Size: int64(len(body))})).To(Succeed())
|
||||
_, e := tw.Write(body)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
}
|
||||
Expect(tw.Close()).To(Succeed())
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
archives["/"+version+".tar"] = b.Bytes()
|
||||
releases = append(releases, catalog.Release{ID: "sample", Name: "Sample", Version: version, OS: "linux", Arch: "amd64",
|
||||
Artifact: url + "/" + version + ".tar", Digest: fmt.Sprintf("sha256:%x", sha256.Sum256(b.Bytes()))})
|
||||
}
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if data, ok := archives[r.URL.Path]; ok {
|
||||
_, _ = w.Write(data)
|
||||
return
|
||||
}
|
||||
_ = catalog.WriteIndex(w, releases)
|
||||
}))
|
||||
DeferCleanup(server.Close)
|
||||
url = server.URL
|
||||
publish("1.2.3")
|
||||
Expect(os.MkdirAll(c.Root, 0700)).To(Succeed())
|
||||
sources, e := catalog.NewSources(filepath.Join(c.Root, "sources.json"), c.AdminCheck, catalog.CredentialResolver{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, catalog.Source{ID: "test", Name: "Test", URL: url, Enabled: true, AllowLoopbackHTTP: true})).To(Succeed())
|
||||
a := &Application{applicationConfig: &config.ApplicationConfig{Context: context.Background(), AutoUpgradeAddons: autoUpgrade}}
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
_, e = a.AddonManager().Submit(context.Background(), contract.Actor{}, addons.LifecycleCommand{OperationID: "install", AddonID: "sample", Action: "install", SourceID: "test", Version: "1.2.3"})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
publish("1.2.4")
|
||||
version := func() string {
|
||||
s, e := a.AddonManager().Status(context.Background(), "sample")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
return s.Generations[len(s.Generations)-1].Receipt.Manifest.Version
|
||||
}
|
||||
return a, version
|
||||
}
|
||||
|
||||
It("is off by default", func() {
|
||||
Expect((&config.ApplicationConfig{}).AutoUpgradeAddons).To(BeFalse())
|
||||
})
|
||||
|
||||
It("leaves an installed add-on alone when the setting is off", func() {
|
||||
a, version := start(false)
|
||||
a.autoUpgradeAddons(context.Background())
|
||||
Expect(version()).To(Equal("1.2.3"))
|
||||
})
|
||||
|
||||
It("upgrades an installed add-on through the manager when the setting is on", func() {
|
||||
a, version := start(true)
|
||||
a.autoUpgradeAddons(context.Background())
|
||||
Expect(version()).To(Equal("1.2.4"))
|
||||
})
|
||||
|
||||
It("follows the setting as it changes at run time", func() {
|
||||
a, version := start(false)
|
||||
a.autoUpgradeAddons(context.Background())
|
||||
Expect(version()).To(Equal("1.2.3"))
|
||||
a.applicationConfig.AutoUpgradeAddons = true
|
||||
a.autoUpgradeAddons(context.Background())
|
||||
Expect(version()).To(Equal("1.2.4"))
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,34 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"slices"
|
||||
|
||||
"github.com/mudler/LocalAI/core/services/addons/hostops"
|
||||
addonruntime "github.com/mudler/LocalAI/core/services/addons/runtime"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// Embedding the interfaces satisfies them; registration never calls a method.
|
||||
type noFineTune struct{ hostops.FineTuner }
|
||||
type noQuant struct{ hostops.Quantizer }
|
||||
type noBackends struct{ hostops.BackendLister }
|
||||
|
||||
var _ = Describe("RegisterAddonEngineJobs", func() {
|
||||
It("registers nothing without a broker", func() {
|
||||
a := &Application{}
|
||||
Expect(a.RegisterAddonEngineJobs(noFineTune{}, noQuant{}, noBackends{})).To(Succeed())
|
||||
})
|
||||
|
||||
It("registers through the broker", func() {
|
||||
b := hostops.New(hostops.NewInFlight(), nil, contract.DefaultLimits())
|
||||
a := &Application{addonHostOps: &addonruntime.HostOps{Broker: b}}
|
||||
Expect(a.RegisterAddonEngineJobs(noFineTune{}, noQuant{}, noBackends{})).To(Succeed())
|
||||
ops := b.Ops()
|
||||
for _, want := range []string{"finetune.start", "finetune.get", "finetune.list", "finetune.checkpoints", "quantize.start", "quantize.get", "quantize.list"} {
|
||||
Expect(slices.Contains(ops, want)).To(BeTrue(), "%s is not registered: %v", want, ops)
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,425 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
mcpTools "github.com/mudler/LocalAI/core/http/endpoints/mcp"
|
||||
"github.com/mudler/LocalAI/core/services/addons"
|
||||
"github.com/mudler/LocalAI/core/services/addons/artifacts"
|
||||
"github.com/mudler/LocalAI/core/services/addons/gateway"
|
||||
"github.com/mudler/LocalAI/core/services/addons/hostops"
|
||||
"github.com/mudler/LocalAI/core/services/addons/policy"
|
||||
"github.com/mudler/LocalAI/core/services/addons/resources"
|
||||
addonruntime "github.com/mudler/LocalAI/core/services/addons/runtime"
|
||||
addontools "github.com/mudler/LocalAI/core/services/addons/tools"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
"github.com/mudler/xlog"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func (a *Application) AddonGateway() *gateway.Gateway { return a.addonGateway }
|
||||
|
||||
// AddonToolProvider binds the tools of installed addons to the principal of a
|
||||
// chat request. It is nil when the host has no addon gateway: the chat
|
||||
// handler then rejects an opt-in with a typed error. The provider reads the
|
||||
// ownership mode from configuration, never from a request.
|
||||
func (a *Application) AddonToolProvider() *mcpTools.AddonToolProvider {
|
||||
if a.addonGateway == nil {
|
||||
return nil
|
||||
}
|
||||
return mcpTools.NewAddonToolProvider(addontools.NewToolAdapter(a.addonGateway), a.AddonOwnershipMode())
|
||||
}
|
||||
|
||||
// AddonHostOps is the host operation channel of the add-on supervisor, or nil
|
||||
// when this process offers none.
|
||||
func (a *Application) AddonHostOps() *addonruntime.HostOps { return a.addonHostOps }
|
||||
|
||||
// newAddonHostOps builds the host operation broker for the supervisor of the
|
||||
// owner. A frontend without the add-on manager, or a host whose setting names
|
||||
// no add-on, builds nothing, so the channel fails closed. The setting is a
|
||||
// host setting read at startup; no manifest or request can add to it.
|
||||
func (a *Application) newAddonHostOps(m *addons.Manager, limits contract.Limits) *addonruntime.HostOps {
|
||||
allowed := a.applicationConfig.AddonHostOps
|
||||
if m == nil || len(allowed) == 0 {
|
||||
return nil
|
||||
}
|
||||
in := hostops.NewInFlight()
|
||||
broker := hostops.New(in, addonFeatureCheck(a.authDB), limits)
|
||||
broker.SetAdminCheck(addonAdminCheck(a.authDB))
|
||||
return &addonruntime.HostOps{Broker: broker, InFlight: in, Allowed: allowed}
|
||||
}
|
||||
|
||||
// RegisterAddonEngineJobs registers the engine job operations with the services
|
||||
// the routes use. The broker is built at startup and the services after it, so
|
||||
// the HTTP setup calls this once they exist. A frontend without the broker has
|
||||
// nothing to register, and the channel then fails closed.
|
||||
func (a *Application) RegisterAddonEngineJobs(ft hostops.FineTuner, q hostops.Quantizer, backends hostops.BackendLister) error {
|
||||
if a.addonHostOps == nil {
|
||||
return nil
|
||||
}
|
||||
return hostops.RegisterEngineJobs(a.addonHostOps.Broker, ft, q, backends)
|
||||
}
|
||||
|
||||
// RegisterAddonEngineProgress registers the progress and stop operations over
|
||||
// the same services as RegisterAddonEngineJobs. The sources are wrapped in the
|
||||
// bounded-wait reader here; a nil source registers nothing for its kind.
|
||||
func (a *Application) RegisterAddonEngineProgress(ft, q hostops.ProgressSource) error {
|
||||
if a.addonHostOps == nil {
|
||||
return nil
|
||||
}
|
||||
var ftp, qp hostops.Progresser
|
||||
if ft != nil {
|
||||
ftp = hostops.NewProgresser(ft)
|
||||
}
|
||||
if q != nil {
|
||||
qp = hostops.NewProgresser(q)
|
||||
}
|
||||
return hostops.RegisterEngineProgress(a.addonHostOps.Broker, ftp, qp)
|
||||
}
|
||||
|
||||
// RegisterAddonEngineModels registers the model registration operations over
|
||||
// the fine-tuning and quantization services. A nil source registers nothing
|
||||
// for its kind.
|
||||
func (a *Application) RegisterAddonEngineModels(ex hostops.Exporter, im hostops.Importer) error {
|
||||
if a.addonHostOps == nil {
|
||||
return nil
|
||||
}
|
||||
return hostops.RegisterEngineModels(a.addonHostOps.Broker, ex, im)
|
||||
}
|
||||
|
||||
// RegisterAddonEngineDatasets registers the inline dataset operation over the
|
||||
// fine-tuning dataset store. A nil store registers nothing.
|
||||
func (a *Application) RegisterAddonEngineDatasets(ds hostops.DatasetStore) error {
|
||||
if a.addonHostOps == nil {
|
||||
return nil
|
||||
}
|
||||
return hostops.RegisterEngineDatasets(a.addonHostOps.Broker, ds)
|
||||
}
|
||||
|
||||
// addonFeatureCheck answers whether an active account holds a feature, with
|
||||
// the same rule as the route middleware. Without an account database, or for
|
||||
// an account that is not active, the answer is no.
|
||||
func addonFeatureCheck(db *gorm.DB) func(userID, feature string) bool {
|
||||
return func(userID, feature string) bool {
|
||||
if db == nil || userID == "" {
|
||||
return false
|
||||
}
|
||||
var u auth.User
|
||||
if db.Where("id = ? AND status = ?", userID, auth.StatusActive).First(&u).Error != nil {
|
||||
return false
|
||||
}
|
||||
return auth.HasFeatureAccess(db, &u, feature)
|
||||
}
|
||||
}
|
||||
|
||||
// addonAdminCheck answers whether an actor is an administrator. With an account
|
||||
// database the role of the active account decides in every mode, with the rule
|
||||
// of the admin routes (auth.RequireAdmin); a local-owner actor is not an
|
||||
// administrator by mode alone. Without an account database there are no roles,
|
||||
// so the local owner of the install is the administrator and no other actor is.
|
||||
func addonAdminCheck(db *gorm.DB) func(actor contract.Actor) bool {
|
||||
return func(actor contract.Actor) bool {
|
||||
if db == nil {
|
||||
return actor.Mode == "local-owner"
|
||||
}
|
||||
if actor.UserID == "" {
|
||||
return false
|
||||
}
|
||||
var u auth.User
|
||||
if db.Where("id = ? AND status = ?", actor.UserID, auth.StatusActive).First(&u).Error != nil {
|
||||
return false
|
||||
}
|
||||
return u.Role == auth.RoleAdmin
|
||||
}
|
||||
}
|
||||
|
||||
// AddonPolicy is the owner-local resource policy. It is nil on frontends and on
|
||||
// hosts without the addon transport, which then deny every resource.
|
||||
func (a *Application) AddonPolicy() *policy.Policy { return a.addonPolicy }
|
||||
|
||||
// AddonOwnershipMode is the configured ownership mode. Named-user is the
|
||||
// default; local-owner exists only when the deployment asked for it.
|
||||
func (a *Application) AddonOwnershipMode() string {
|
||||
if m := a.applicationConfig.Addons.OwnershipMode; m != "" {
|
||||
return m
|
||||
}
|
||||
return auth.AddonModeNamedUser
|
||||
}
|
||||
|
||||
// addonStatus is the manager view that creation authority consults.
|
||||
type addonStatus interface {
|
||||
Status(ctx context.Context, id string) (addons.Status, error)
|
||||
}
|
||||
|
||||
// addonCreateAuthority lets an actor create a resource only for an addon that
|
||||
// is installed and enabled on the owner, with no lifecycle change in flight.
|
||||
// A manager that cannot answer is unavailable, never a grant.
|
||||
func addonCreateAuthority(status addonStatus, revalidate func(context.Context, contract.Actor) error) func(context.Context, contract.Actor, string) error {
|
||||
return func(ctx context.Context, actor contract.Actor, addonID string) error {
|
||||
if status == nil || revalidate == nil {
|
||||
return gateway.ErrForbidden
|
||||
}
|
||||
if err := revalidate(ctx, actor); err != nil {
|
||||
return err
|
||||
}
|
||||
s, err := status.Status(ctx, addonID)
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
return fmt.Errorf("addon status: %w: %w", gateway.ErrUnavailable, err)
|
||||
}
|
||||
if !s.Installed || !s.DesiredEnabled || !s.StartEligible {
|
||||
return gateway.ErrForbidden
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func (a *Application) startAddonGateway() error {
|
||||
cfg := a.applicationConfig.Addons
|
||||
if cfg.Transport == nil {
|
||||
return nil
|
||||
}
|
||||
mode := a.AddonOwnershipMode()
|
||||
if mode == auth.AddonModeNamedUser && a.authDB == nil {
|
||||
return gateway.ErrForbidden
|
||||
}
|
||||
c := *cfg.Transport
|
||||
if c.InstanceID != cfg.InstanceID || c.OwnerID != cfg.OwnerID || c.OwnerAddress != cfg.OwnerAddress {
|
||||
return gateway.ErrForbidden
|
||||
}
|
||||
carrier, e := gateway.NewCarrier(c)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
if mode == auth.AddonModeLocalOwner {
|
||||
xlog.Warn(localOwnerWarning)
|
||||
}
|
||||
g := &gateway.Gateway{Limits: c.Limits, Manager: a.addonManager, Revalidate: auth.AddonRevalidate(a.authDB, mode)}
|
||||
if cfg.InstanceID != cfg.OwnerID {
|
||||
g.Forward = carrier
|
||||
a.addonGateway = g
|
||||
return nil
|
||||
}
|
||||
// Only the owner holds the policy store. Other frontends reach it through the
|
||||
// gateway, and without it the gateway keeps denying every resource.
|
||||
if !filepath.IsAbs(cfg.Root) || cfg.ListenAddress == "" {
|
||||
return gateway.ErrForbidden
|
||||
}
|
||||
store, e := policy.OpenStore(filepath.Join(cfg.Root, "policy"), policy.Mode(mode))
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
// Proposals are durable beside the policy store and held only here. The
|
||||
// broker is the policy's proposal invalidator, so a revocation cancels the
|
||||
// pending proposals of that user before it is acknowledged.
|
||||
brokerOpts := policy.BrokerOptions{}
|
||||
if a.addonManager != nil {
|
||||
brokerOpts.Runtime = a.addonManager
|
||||
}
|
||||
// The same existence check settles the registrations a crash left and the
|
||||
// reservations of create proposals the proposer stops waiting for.
|
||||
exists := a.addonResourceExists
|
||||
if exists == nil {
|
||||
exists = a.addonBackingExists(contract.DefaultLimits())
|
||||
}
|
||||
brokerOpts.BackingExists = exists
|
||||
// A deletion trusts only an explicit answer of the add-on.
|
||||
confirmed := a.addonResourceExists
|
||||
if confirmed == nil {
|
||||
confirmed = a.addonBackingConfirmed(contract.DefaultLimits())
|
||||
}
|
||||
broker, e := policy.OpenBroker(filepath.Join(cfg.Root, "proposals"), brokerOpts)
|
||||
if e != nil {
|
||||
_ = store.Close()
|
||||
return e
|
||||
}
|
||||
p := policy.New(store, policy.Options{
|
||||
Revalidate: g.Revalidate,
|
||||
CreateAuthority: addonCreateAuthority(a.addonManager, g.Revalidate),
|
||||
Proposals: broker,
|
||||
})
|
||||
broker.Attach(p)
|
||||
// Recovery gets the owner's access state. A restore or a purge that a
|
||||
// crash stopped is finished now, before anything is served; an add-on
|
||||
// that needs recovery stays unavailable until an administrator acts.
|
||||
if a.addonManager != nil {
|
||||
if rec := a.addonManager.DataRecovery(); rec != nil {
|
||||
rec.SetPolicy(&policy.RecoveryPort{Policy: p, Broker: broker})
|
||||
if e = rec.Reconcile(a.applicationConfig.Context); e != nil {
|
||||
xlog.Warn("addons: a data recovery journal could not be finished yet", "error", e)
|
||||
}
|
||||
}
|
||||
}
|
||||
var files *artifacts.Artifacts
|
||||
var selections *resources.SelectionStore
|
||||
closeState := func() {
|
||||
if files != nil {
|
||||
_ = files.Close()
|
||||
}
|
||||
if selections != nil {
|
||||
_ = selections.Close()
|
||||
}
|
||||
if a.addonReconcile != nil {
|
||||
a.addonReconcile.Close()
|
||||
}
|
||||
_ = broker.Close()
|
||||
_ = store.Close()
|
||||
}
|
||||
// Registrations left pending by a crash are settled before anything is
|
||||
// served. A record whose backing cannot be confirmed now (its addon is
|
||||
// disabled or failed to start) stays pending and hidden; it never stops
|
||||
// the host from starting, and the loop below settles it later.
|
||||
rep, e := p.ReconcileReport(a.applicationConfig.Context, exists)
|
||||
if e != nil {
|
||||
closeState()
|
||||
return errors.Join(e, addons.ErrRecoveryRequired)
|
||||
}
|
||||
if rep.Kept > 0 {
|
||||
xlog.Warn("addons: pending resource registrations could not be settled yet and stay pending", "settled", rep.Settled, "kept", rep.Kept)
|
||||
}
|
||||
// The conversation selections and the UI-only resource management sit
|
||||
// beside the policy store and are held only here.
|
||||
selections, e = resources.OpenSelections(filepath.Join(cfg.Root, "selections"), resources.SelectionOptions{})
|
||||
if e != nil {
|
||||
closeState()
|
||||
return errors.Join(e, addons.ErrRecoveryRequired)
|
||||
}
|
||||
resourceOpts := resources.Options{Policy: p, Broker: broker, Users: auth.NewAddonUsers(a.authDB), Selections: selections, BackingConfirmed: confirmed, Limits: contract.DefaultLimits()}
|
||||
if a.addonManager != nil {
|
||||
resourceOpts.Runtime, resourceOpts.Capabilities, resourceOpts.Tools = a.addonManager, a.addonManager, a.addonManager
|
||||
}
|
||||
manage := resources.New(resourceOpts)
|
||||
g.Resources = manage
|
||||
loop := policy.NewReconcileLoop(p, exists, a.addonReconcileTick)
|
||||
loop.Resume = manage.Resume
|
||||
loop.Report = func(r policy.ReconcileReport, err error) {
|
||||
if err != nil || r.Kept > 0 {
|
||||
xlog.Warn("addons: pending resource registrations remain", "settled", r.Settled, "kept", r.Kept, "error", err)
|
||||
}
|
||||
}
|
||||
// A deletion a crash left is resumed before anything is served; one that
|
||||
// cannot be finished now stays inaccessible and the loop retries it.
|
||||
manage.Resume(a.applicationConfig.Context)
|
||||
loop.Start(a.applicationConfig.Context)
|
||||
a.addonReconcile = loop
|
||||
// A proposal that was executing when the owner stopped is settled by
|
||||
// receipt lookup, or recorded as outcome-unknown. Nothing is applied again.
|
||||
if e = broker.Recover(a.applicationConfig.Context); e != nil {
|
||||
closeState()
|
||||
return errors.Join(e, addons.ErrRecoveryRequired)
|
||||
}
|
||||
// Unresolved outcomes are looked up again with backoff for a finite
|
||||
// window, and expiry stays recorded. Run ends with the application
|
||||
// context or when the broker is closed.
|
||||
go broker.Run(a.applicationConfig.Context)
|
||||
provider := p.Gateway()
|
||||
g.Admission, g.Resolver = provider, provider
|
||||
g.Proposals = broker.Gateway()
|
||||
// Private file results are staged beside the other owner state. Without a
|
||||
// manager there is no runtime to read them from, and the gateway then
|
||||
// denies every file result and download.
|
||||
if a.addonManager != nil {
|
||||
files, e = artifacts.Open(artifacts.Options{
|
||||
Dir: filepath.Join(cfg.Root, "artifacts"),
|
||||
Policy: p,
|
||||
Runtime: a.addonManager,
|
||||
Addon: artifactAddonState(a.addonManager),
|
||||
Limits: artifacts.DefaultLimits().WithExpiry(cfg.ArtifactExpiry),
|
||||
})
|
||||
if e != nil {
|
||||
closeState()
|
||||
return e
|
||||
}
|
||||
provider.Outputs = files
|
||||
g.Artifacts = files.Gateway()
|
||||
go files.Run(a.applicationConfig.Context)
|
||||
}
|
||||
handler, e := carrier.OwnerHandler(g)
|
||||
if e != nil {
|
||||
closeState()
|
||||
return e
|
||||
}
|
||||
listener, e := net.Listen("tcp", cfg.ListenAddress)
|
||||
if e != nil {
|
||||
closeState()
|
||||
return e
|
||||
}
|
||||
server := &http.Server{Handler: handler, TLSConfig: carrier.ServerTLS(), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second, WriteTimeout: 65 * time.Second, IdleTimeout: 10 * time.Second, MaxHeaderBytes: 16 << 10}
|
||||
a.addonServer = server
|
||||
a.addonGateway = g
|
||||
a.addonPolicy, a.addonStore = p, store
|
||||
if pinStore, err := a.AddonPinStore(); err != nil {
|
||||
xlog.Warn("addon pin sweep not started", "error", err)
|
||||
} else {
|
||||
a.startAddonPinSweep(a.applicationConfig.Context, pinStore)
|
||||
}
|
||||
go func() { _ = server.Serve(tls.NewListener(listener, server.TLSConfig)) }()
|
||||
go func() { <-a.applicationConfig.Context.Done(); _ = server.Close(); closeState() }()
|
||||
return nil
|
||||
}
|
||||
|
||||
// addonBackingExists is the real existence check for resources whose backing
|
||||
// lives in an addon (Tables databases). Without a manager there is no addon
|
||||
// that could back a resource, so a pending record is dropped as before.
|
||||
func (a *Application) addonBackingExists(limits contract.Limits) func(context.Context, policy.ResourceKey) (bool, error) {
|
||||
m := a.addonManager
|
||||
if m == nil {
|
||||
return func(context.Context, policy.ResourceKey) (bool, error) { return false, nil }
|
||||
}
|
||||
return policy.BackingExists(policy.BackingOptions{
|
||||
Runtime: m,
|
||||
Tools: m,
|
||||
Installed: func(ctx context.Context, id string) (bool, error) {
|
||||
s, err := m.Status(ctx, id)
|
||||
return s.Installed, err
|
||||
},
|
||||
Limits: limits,
|
||||
})
|
||||
}
|
||||
|
||||
// addonBackingConfirmed is the existence check of a deletion. Unlike
|
||||
// addonBackingExists it never reads a missing manager, an add-on that is not
|
||||
// installed or one that does not declare the existence tool as "gone".
|
||||
func (a *Application) addonBackingConfirmed(limits contract.Limits) func(context.Context, policy.ResourceKey) (bool, error) {
|
||||
m := a.addonManager
|
||||
if m == nil {
|
||||
return func(context.Context, policy.ResourceKey) (bool, error) { return false, policy.ErrBackingUnanswered }
|
||||
}
|
||||
return policy.BackingConfirmed(policy.BackingOptions{
|
||||
Runtime: m,
|
||||
Tools: m,
|
||||
Installed: func(ctx context.Context, id string) (bool, error) {
|
||||
s, err := m.Status(ctx, id)
|
||||
return s.Installed, err
|
||||
},
|
||||
Limits: limits,
|
||||
})
|
||||
}
|
||||
|
||||
// artifactAddonState reports the lifecycle state the artifact store needs. A
|
||||
// manager that cannot answer is an error, never an enabled addon.
|
||||
func artifactAddonState(status addonStatus) func(context.Context, string) (artifacts.AddonState, error) {
|
||||
return func(ctx context.Context, addonID string) (artifacts.AddonState, error) {
|
||||
s, err := status.Status(ctx, addonID)
|
||||
if err != nil {
|
||||
return artifacts.AddonState{}, err
|
||||
}
|
||||
return artifacts.AddonState{Installed: s.Installed, Enabled: s.DesiredEnabled, Generation: s.Generation}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// localOwnerWarning is logged at startup in local-owner mode. There is no
|
||||
// session in that mode, so the confirmation boundary is the browser headers
|
||||
// only and does not stop a script that sets them.
|
||||
const localOwnerWarning = "addons: local-owner mode is enabled; unauthenticated callers share one authority over every addon resource and sharing is disabled; any caller that can reach this port can confirm addon proposals"
|
||||
@@ -0,0 +1,32 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
//go:build auth
|
||||
|
||||
package application
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("the feature check", func() {
|
||||
It("follows the account permissions", func() {
|
||||
db, err := auth.InitDB(filepath.Join(GinkgoT().TempDir(), "auth.db"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
mk := func(id, role, status string) {
|
||||
Expect(db.Create(&auth.User{ID: id, Email: id + "@example.com", Role: role, Status: status}).Error).To(Succeed())
|
||||
}
|
||||
mk("admin", auth.RoleAdmin, auth.StatusActive)
|
||||
mk("plain", auth.RoleUser, auth.StatusActive)
|
||||
mk("granted", auth.RoleUser, auth.StatusActive)
|
||||
mk("gone", auth.RoleAdmin, "disabled")
|
||||
Expect(auth.UpdateUserPermissions(db, "granted", auth.PermissionMap{"fine_tuning": true})).To(Succeed())
|
||||
Expect(auth.UpdateUserPermissions(db, "plain", auth.PermissionMap{"fine_tuning": false})).To(Succeed())
|
||||
has := addonFeatureCheck(db)
|
||||
for user, want := range map[string]bool{"admin": true, "granted": true, "plain": false, "gone": false, "nobody": false, "": false} {
|
||||
Expect(has(user, "fine_tuning")).To(Equal(want), "user %q", user)
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,29 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("add-on host operations", func() {
|
||||
It("are not offered by a frontend without the add-on process", func() {
|
||||
named := map[string][]string{"fine-tuning": {"finetune."}}
|
||||
a := &Application{applicationConfig: &config.ApplicationConfig{AddonHostOps: named}}
|
||||
Expect(a.newAddonHostOps(nil, contract.DefaultLimits())).To(BeNil(), "a process without the add-on manager built a broker")
|
||||
Expect(a.AddonHostOps()).To(BeNil(), "a process that never started the supervisor offers host operations")
|
||||
})
|
||||
|
||||
It("are not offered by a host that names no add-on", func() {
|
||||
for _, ops := range []map[string][]string{nil, {}} {
|
||||
a := &Application{applicationConfig: &config.ApplicationConfig{AddonHostOps: ops}}
|
||||
Expect(a.newAddonHostOps(nil, contract.DefaultLimits())).To(BeNil(), "%v built a broker", ops)
|
||||
}
|
||||
})
|
||||
|
||||
It("refuses the feature check without an account database", func() {
|
||||
Expect(addonFeatureCheck(nil)("u1", "fine_tuning")).To(BeFalse(), "no database must mean no feature")
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,34 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log/slog"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/mudler/xlog"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func capture() *bytes.Buffer {
|
||||
var buf bytes.Buffer
|
||||
xlog.SetLogger(xlog.NewLoggerWithHandler(slog.NewTextHandler(&buf, &slog.HandlerOptions{Level: slog.LevelDebug}), xlog.LogLevelDebug))
|
||||
return &buf
|
||||
}
|
||||
|
||||
var _ = Describe("pinStoreFor", func() {
|
||||
It("logs one warning for memory-only pins", func() {
|
||||
buf := capture()
|
||||
_, err := pinStoreFor(nil, "")
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(bytes.Count(buf.Bytes(), []byte("memory only"))).To(Equal(1), "log: %q", buf.String())
|
||||
})
|
||||
|
||||
It("does not warn with a pins file", func() {
|
||||
buf := capture()
|
||||
_, err := pinStoreFor(nil, filepath.Join(GinkgoT().TempDir(), "addon-pins.json"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(buf.Len()).To(BeZero(), "unexpected log: %q", buf.String())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,64 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"slices"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/core/services/addons/builtin"
|
||||
"github.com/mudler/LocalAI/core/services/addons/pins"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
type fakeOwner struct {
|
||||
ids []string
|
||||
err error
|
||||
}
|
||||
|
||||
func (f fakeOwner) InstalledIDs(context.Context) ([]string, error) { return f.ids, f.err }
|
||||
|
||||
func pinnedIDs(s pins.Store, user string) []string {
|
||||
list, err := s.List(context.Background(), user)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
out := []string{}
|
||||
for _, p := range list {
|
||||
out = append(out, p.AddonID)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
var _ = Describe("sweepAddonPins", func() {
|
||||
It("never deletes the built-in pins", func() {
|
||||
// No page is built in today, so the test makes up two.
|
||||
DeferCleanup(builtin.SetCatalogForTest([]builtin.Entry{
|
||||
{ID: "guide", Kind: builtin.KindPage, TitleKey: "items.chat", Icon: "chat", MainPath: "/app/chat"},
|
||||
{ID: "help", Kind: builtin.KindPage, TitleKey: "items.models", Icon: "boxes", MainPath: "/app/models"},
|
||||
}))
|
||||
ctx := context.Background()
|
||||
s, _ := pins.NewFileStore("")
|
||||
_ = s.Replace(ctx, "u1", []string{"guide", "help", "notes"})
|
||||
t0 := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||
owner := fakeOwner{ids: []string{}}
|
||||
sweepAddonPins(ctx, s, owner, t0)
|
||||
sweepAddonPins(ctx, s, owner, t0.Add(pins.Retention+time.Hour))
|
||||
// notes is not installed and expired; the built-ins are always present.
|
||||
got := pinnedIDs(s, "u1")
|
||||
Expect(slices.Equal(got, []string{"guide", "help"})).To(BeTrue(), "unexpected pins %v", got)
|
||||
})
|
||||
|
||||
It("is skipped when the owner does not answer", func() {
|
||||
ctx := context.Background()
|
||||
s, _ := pins.NewFileStore("")
|
||||
_ = s.Replace(ctx, "u1", []string{"notes"})
|
||||
t0 := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||
owner := fakeOwner{err: errors.New("owner unavailable")}
|
||||
sweepAddonPins(ctx, s, owner, t0)
|
||||
sweepAddonPins(ctx, s, owner, t0.Add(pins.Retention+time.Hour))
|
||||
list, _ := s.List(ctx, "u1")
|
||||
Expect(list).To(HaveLen(1), "a sweep without an answer must neither mark nor delete: %+v", list)
|
||||
Expect(list[0].MissingSince).To(BeNil())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,136 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
"github.com/mudler/LocalAI/core/services/addons/builtin"
|
||||
"github.com/mudler/LocalAI/core/services/addons/pins"
|
||||
"github.com/mudler/LocalAI/core/services/advisorylock"
|
||||
"github.com/mudler/xlog"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// addonPinSweepEvery is how often the retention sweep runs.
|
||||
const addonPinSweepEvery = 6 * time.Hour
|
||||
|
||||
// addonPinSeedTimeout bounds the one-time upgrade seed at start.
|
||||
const addonPinSeedTimeout = 30 * time.Second
|
||||
|
||||
// AddonPinStore returns the pin store, creating it once so the routes and the
|
||||
// retention sweep share it. Creating it also runs the one-time upgrade seed for
|
||||
// the accounts that exist now. An error is cached: a store that could not be
|
||||
// opened stays unavailable, and startup must stop rather than fall back to
|
||||
// memory and lose every unpin at the next restart.
|
||||
func (a *Application) AddonPinStore() (pins.Store, error) {
|
||||
a.addonPinStoreOnce.Do(func() {
|
||||
s, err := pinStoreFor(a.authDB, a.applicationConfig.DataPath)
|
||||
if err != nil {
|
||||
a.addonPinStoreErr = err
|
||||
return
|
||||
}
|
||||
a.addonPinStore = s
|
||||
// The seed gets a short bound so a stuck database cannot hold startup.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), addonPinSeedTimeout)
|
||||
defer cancel()
|
||||
accounts, err := a.existingPinAccounts()
|
||||
if err != nil {
|
||||
// No marker is written: the seed runs again at the next start, so a
|
||||
// failed read does not cost the existing accounts their pins.
|
||||
xlog.Warn("addon pin upgrade seed skipped; it will run again at the next start", "error", err)
|
||||
return
|
||||
}
|
||||
if _, err := s.SeedExisting(ctx, accounts, pins.SeedIDs); err != nil {
|
||||
xlog.Warn("addon pin upgrade seed failed; it will run again at the next start", "error", err)
|
||||
}
|
||||
})
|
||||
return a.addonPinStore, a.addonPinStoreErr
|
||||
}
|
||||
|
||||
// existingPinAccounts are the keys pins are stored under for the accounts that
|
||||
// exist at this moment: every user of the account database, or the one local
|
||||
// owner when authentication is off. The local owner counts as an existing
|
||||
// account, so a deployment with authentication off keeps its Studio and Talk
|
||||
// entries too.
|
||||
func (a *Application) existingPinAccounts() ([]string, error) {
|
||||
if a.authDB == nil {
|
||||
return []string{auth.AddonLocalOwnerID}, nil
|
||||
}
|
||||
var ids []string
|
||||
if err := a.authDB.Model(&auth.User{}).Pluck("id", &ids).Error; err != nil {
|
||||
return nil, fmt.Errorf("list the accounts for the pin seed: %w", err)
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// pinStoreFor picks the store: the shared account database when there is one,
|
||||
// otherwise a file next to the other add-on data (one local owner, one host).
|
||||
func pinStoreFor(db *gorm.DB, dataPath string) (pins.Store, error) {
|
||||
if db != nil {
|
||||
s, err := pins.NewGormStore(db)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open the add-on pin table: %w", err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
path := ""
|
||||
if dataPath != "" {
|
||||
path = filepath.Join(dataPath, "addon-pins.json")
|
||||
} else {
|
||||
xlog.Warn("No data path and no account database: add-on pins are kept in memory only and do not survive a restart")
|
||||
}
|
||||
s, err := pins.NewFileStore(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open the add-on pins file %q: %w", path, err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// startAddonPinSweep expires the pins of removed add-ons. With an account
|
||||
// database one frontend at a time runs it (a leader loop). Without one there
|
||||
// is one process and a plain ticker runs it.
|
||||
func (a *Application) startAddonPinSweep(ctx context.Context, store pins.Store) {
|
||||
gw := a.addonGateway
|
||||
if gw == nil || gw.Manager == nil || store == nil {
|
||||
return
|
||||
}
|
||||
run := func() { sweepAddonPins(ctx, store, gw.Manager, time.Now().UTC()) }
|
||||
if db := a.authDB; db != nil {
|
||||
go advisorylock.RunLeaderLoop(ctx, db, advisorylock.KeyAddonPinSweep, addonPinSweepEvery, run)
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
t := time.NewTicker(addonPinSweepEvery)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-t.C:
|
||||
run()
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// installedLister is the part of the add-on manager the sweep reads.
|
||||
type installedLister interface {
|
||||
InstalledIDs(ctx context.Context) ([]string, error)
|
||||
}
|
||||
|
||||
// sweepAddonPins runs one retention sweep. The built-in IDs are always present.
|
||||
// When the owner cannot answer, nothing happens: an outage must not expire pins.
|
||||
func sweepAddonPins(ctx context.Context, store pins.Store, owner installedLister, now time.Time) {
|
||||
ids, err := owner.InstalledIDs(ctx)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
present := append(ids, builtin.IDs()...)
|
||||
if _, err := pins.Sweep(ctx, store, present, now); err != nil {
|
||||
xlog.Warn("addon pin sweep failed", "error", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
//go:build auth
|
||||
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
"github.com/mudler/LocalAI/core/services/addons/pins"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
var _ = Describe("add-on pin store and the upgrade seed", func() {
|
||||
pinIDs := func(s pins.Store, user string) []string {
|
||||
list, err := s.List(context.Background(), user)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
ids := []string{}
|
||||
for _, p := range list {
|
||||
ids = append(ids, p.AddonID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
appOver := func(db *gorm.DB, dataPath string) *Application {
|
||||
return &Application{
|
||||
applicationConfig: &config.ApplicationConfig{Context: context.Background(), DataPath: dataPath},
|
||||
authDB: db,
|
||||
}
|
||||
}
|
||||
store := func(a *Application) pins.Store {
|
||||
s, err := a.AddonPinStore()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
return s
|
||||
}
|
||||
|
||||
It("pins studio and talk once for the accounts that exist and never again", func() {
|
||||
db, err := auth.InitDB(filepath.Join(GinkgoT().TempDir(), "auth.db"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
for _, id := range []string{"u1", "u2"} {
|
||||
Expect(db.Create(&auth.User{ID: id, Email: id + "@example.test", Provider: auth.ProviderLocal, Role: auth.RoleUser, Status: auth.StatusActive}).Error).To(Succeed())
|
||||
}
|
||||
a := appOver(db, "")
|
||||
s := store(a)
|
||||
Expect(pinIDs(s, "u1")).To(Equal([]string{"studio", "talk"}))
|
||||
Expect(pinIDs(s, "u2")).To(Equal([]string{"studio", "talk"}))
|
||||
|
||||
// An account created later starts with no pins, and a second seed does nothing.
|
||||
Expect(db.Create(&auth.User{ID: "u3", Email: "u3@example.test", Provider: auth.ProviderLocal, Role: auth.RoleUser, Status: auth.StatusActive}).Error).To(Succeed())
|
||||
Expect(pinIDs(s, "u3")).To(BeEmpty())
|
||||
ran, err := s.SeedExisting(context.Background(), []string{"u3"}, pins.SeedIDs)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(ran).To(BeFalse())
|
||||
|
||||
// The store is created once per application.
|
||||
Expect(store(a)).To(BeIdenticalTo(s))
|
||||
|
||||
// An unpin sticks across a restart or a second frontend over the same database.
|
||||
Expect(s.Replace(context.Background(), "u1", []string{"talk"})).To(Succeed())
|
||||
again := store(appOver(db, ""))
|
||||
Expect(pinIDs(again, "u1")).To(Equal([]string{"talk"}))
|
||||
Expect(pinIDs(again, "u3")).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("does not mark the seed as run when the accounts cannot be read, and seeds at the next start", func() {
|
||||
db, err := auth.InitDB(filepath.Join(GinkgoT().TempDir(), "auth.db"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(db.Create(&auth.User{ID: "u1", Email: "u1@example.test", Provider: auth.ProviderLocal, Role: auth.RoleUser, Status: auth.StatusActive}).Error).To(Succeed())
|
||||
Expect(db.Migrator().RenameTable(&auth.User{}, "users_away")).To(Succeed())
|
||||
|
||||
s := store(appOver(db, ""))
|
||||
Expect(pinIDs(s, "u1")).To(BeEmpty())
|
||||
var marks int64
|
||||
Expect(db.Model(&pins.SeedRun{}).Count(&marks).Error).To(Succeed())
|
||||
Expect(marks).To(BeZero())
|
||||
|
||||
Expect(db.Migrator().RenameTable("users_away", &auth.User{})).To(Succeed())
|
||||
next := store(appOver(db, ""))
|
||||
Expect(pinIDs(next, "u1")).To(Equal([]string{"studio", "talk"}))
|
||||
})
|
||||
|
||||
It("seeds the local owner when authentication is off and keeps an unpin", func() {
|
||||
dir := GinkgoT().TempDir()
|
||||
s := store(appOver(nil, dir))
|
||||
Expect(pinIDs(s, auth.AddonLocalOwnerID)).To(Equal([]string{"studio", "talk"}))
|
||||
Expect(s.Replace(context.Background(), auth.AddonLocalOwnerID, []string{"talk"})).To(Succeed())
|
||||
|
||||
again := store(appOver(nil, dir))
|
||||
Expect(pinIDs(again, auth.AddonLocalOwnerID)).To(Equal([]string{"talk"}))
|
||||
raw, err := os.ReadFile(filepath.Join(dir, "addon-pins.json"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(string(raw)).To(ContainSubstring(pins.UpgradeSeedName))
|
||||
})
|
||||
|
||||
It("reports a corrupt pins file instead of falling back to memory", func() {
|
||||
dir := GinkgoT().TempDir()
|
||||
Expect(os.WriteFile(filepath.Join(dir, "addon-pins.json"), []byte("{broken"), 0o600)).To(Succeed())
|
||||
_, err := appOver(nil, dir).AddonPinStore()
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,338 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
//go:build auth
|
||||
|
||||
package application
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/pem"
|
||||
"errors"
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
"github.com/mudler/LocalAI/core/services/addons"
|
||||
"github.com/mudler/LocalAI/core/services/addons/gateway"
|
||||
"github.com/mudler/LocalAI/core/services/addons/policy"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
"github.com/mudler/LocalAI/pkg/system"
|
||||
"github.com/mudler/xlog"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
"gorm.io/gorm"
|
||||
"log/slog"
|
||||
"math/big"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ownerTransport builds the smallest real transport identity of an owner: one
|
||||
// pinned certificate, one signing key and a loopback listen address.
|
||||
func ownerTransport() *gateway.CarrierConfig {
|
||||
pub, key, e := ed25519.GenerateKey(rand.Reader)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
ca := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "test CA"}, NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign}
|
||||
der, e := x509.CreateCertificate(rand.Reader, ca, ca, pub, key)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
ca, e = x509.ParseCertificate(der)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
pool := x509.NewCertPool()
|
||||
pool.AddCert(ca)
|
||||
lp, lk, e := ed25519.GenerateKey(rand.Reader)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
leaf := &x509.Certificate{SerialNumber: big.NewInt(2), Subject: pkix.Name{CommonName: "owner"}, NotBefore: ca.NotBefore, NotAfter: ca.NotAfter, IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}, KeyUsage: x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth}}
|
||||
d, e := x509.CreateCertificate(rand.Reader, leaf, ca, lp, key)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
kd, e := x509.MarshalPKCS8PrivateKey(lk)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
pair, e := tls.X509KeyPair(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: d}), pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: kd}))
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
sp, sk, e := ed25519.GenerateKey(rand.Reader)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
return &gateway.CarrierConfig{InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", Certificate: pair, Roots: pool,
|
||||
Pins: map[string][32]byte{"owner": sha256.Sum256(d)}, SigningKey: sk, VerificationKeys: map[string]ed25519.PublicKey{"owner": sp}}
|
||||
}
|
||||
|
||||
type logCapture struct {
|
||||
mu sync.Mutex
|
||||
buf bytes.Buffer
|
||||
}
|
||||
|
||||
func (l *logCapture) Write(p []byte) (int, error) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
return l.buf.Write(p)
|
||||
}
|
||||
func (l *logCapture) String() string { l.mu.Lock(); defer l.mu.Unlock(); return l.buf.String() }
|
||||
|
||||
var _ = Describe("addon policy bootstrap", func() {
|
||||
setup := func() *Application {
|
||||
return &Application{applicationConfig: &config.ApplicationConfig{Context: context.Background()}}
|
||||
}
|
||||
base := func() config.AddonConfig {
|
||||
c := config.AddonConfig{Enabled: true, InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", StorageID: "local", Root: filepath.Join(realTempDir(), "addons"), ListenAddress: "127.0.0.1:0", Transport: ownerTransport()}
|
||||
return c
|
||||
}
|
||||
authDB := func() *gorm.DB {
|
||||
db, e := auth.InitDB(filepath.Join(GinkgoT().TempDir(), "auth.db"))
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
return db
|
||||
}
|
||||
start := func(a *Application, c config.AddonConfig) error {
|
||||
a.applicationConfig.Addons = c
|
||||
return a.startAddons()
|
||||
}
|
||||
namedActor := func(db *gorm.DB) contract.Actor {
|
||||
Expect(db.Create(&auth.User{ID: "alice", Role: auth.RoleAdmin, Status: auth.StatusActive}).Error).To(Succeed())
|
||||
_, key, e := auth.CreateAPIKey(db, "alice", "test", auth.RoleAdmin, "secret", nil)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
return contract.Actor{UserID: "alice", CredentialID: "key:" + key.ID, Mode: "named-user"}
|
||||
}
|
||||
policyDir := func(c config.AddonConfig) string { return filepath.Join(c.Root, "policy") }
|
||||
|
||||
It("opens the policy on the owner and keeps frontends and hosts without it denying", func() {
|
||||
requireAddonRuntime()
|
||||
c := base()
|
||||
a := setup()
|
||||
a.authDB = authDB()
|
||||
Expect(start(a, c)).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
Expect(a.AddonGateway()).NotTo(BeNil())
|
||||
Expect(a.AddonGateway().Admission).NotTo(BeNil())
|
||||
Expect(a.AddonGateway().Resolver).NotTo(BeNil())
|
||||
Expect(a.AddonPolicy()).NotTo(BeNil())
|
||||
// The owner holds the store: a second opener is refused.
|
||||
_, e := policy.OpenStore(policyDir(c), policy.ModeNamedUser)
|
||||
Expect(e).To(MatchError(policy.ErrStoreBusy))
|
||||
// Unknown resources deny through the installed provider.
|
||||
_, e = a.AddonGateway().Dispatch(context.Background(), namedActor(a.authDB), contract.Request{AddonID: "sample", ResourceID: "nope", OperationID: "op", Generation: 1, Action: "read", Arguments: []byte(`{}`)})
|
||||
Expect(errors.Is(e, gateway.ErrForbidden)).To(BeTrue())
|
||||
|
||||
// A frontend forwards to the owner and never opens or even creates the store.
|
||||
f := base()
|
||||
f.InstanceID = "frontend"
|
||||
f.Transport.InstanceID = "frontend"
|
||||
f.Transport.Pins["frontend"] = f.Transport.Pins["owner"]
|
||||
f.Root = filepath.Join(realTempDir(), "frontend")
|
||||
b := setup()
|
||||
b.authDB = authDB()
|
||||
Expect(start(b, f)).To(Succeed())
|
||||
DeferCleanup(b.Shutdown)
|
||||
Expect(b.AddonGateway()).NotTo(BeNil())
|
||||
Expect(b.AddonGateway().Admission).To(BeNil())
|
||||
Expect(b.AddonGateway().Resolver).To(BeNil())
|
||||
Expect(b.AddonPolicy()).To(BeNil())
|
||||
Expect(policyDir(f)).NotTo(BeADirectory())
|
||||
|
||||
// A host with no transport has no gateway and no policy.
|
||||
n := base()
|
||||
n.Transport = nil
|
||||
n.AdminCheck = func(context.Context, contract.Actor) error { return nil }
|
||||
n.Root = filepath.Join(realTempDir(), "plain")
|
||||
h := setup()
|
||||
Expect(start(h, n)).To(Succeed())
|
||||
DeferCleanup(h.Shutdown)
|
||||
Expect(h.AddonGateway()).To(BeNil())
|
||||
Expect(h.AddonPolicy()).To(BeNil())
|
||||
Expect(policyDir(n)).NotTo(BeADirectory())
|
||||
})
|
||||
|
||||
It("requires the addon to be installed and enabled before a resource can be registered", func() {
|
||||
requireAddonRuntime()
|
||||
c := base()
|
||||
a := setup()
|
||||
a.authDB = authDB()
|
||||
Expect(start(a, c)).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
actor := namedActor(a.authDB)
|
||||
// An addon that was never installed cannot own resources.
|
||||
_, e := a.AddonPolicy().Register(context.Background(), actor, "ghost")
|
||||
Expect(errors.Is(e, gateway.ErrForbidden)).To(BeTrue())
|
||||
// The decision follows the manager's installed and enabled state.
|
||||
status := &statusFixture{}
|
||||
authority := addonCreateAuthority(status, func(context.Context, contract.Actor) error { return nil })
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
status addons.Status
|
||||
err error
|
||||
want int
|
||||
}{
|
||||
{"absent", addons.Status{}, nil, 403},
|
||||
{"installed but disabled", addons.Status{Installed: true}, nil, 403},
|
||||
{"enabled", addons.Status{Installed: true, DesiredEnabled: true, StartEligible: true}, nil, 0},
|
||||
{"change in progress", addons.Status{Installed: true, DesiredEnabled: true}, nil, 403},
|
||||
{"journal needs recovery", addons.Status{}, addons.ErrRecoveryRequired, 503},
|
||||
} {
|
||||
status.s, status.err = row.status, row.err
|
||||
err := authority(context.Background(), actor, "sample")
|
||||
if row.want == 0 {
|
||||
Expect(err).NotTo(HaveOccurred(), row.name)
|
||||
continue
|
||||
}
|
||||
Expect(err).To(HaveOccurred(), row.name)
|
||||
Expect(gateway.StatusFor(err)).To(Equal(row.want), row.name)
|
||||
}
|
||||
Expect(addonCreateAuthority(nil, func(context.Context, contract.Actor) error { return nil })(context.Background(), actor, "sample")).NotTo(Succeed())
|
||||
})
|
||||
|
||||
It("reconciles registrations left pending before the owner starts serving", func() {
|
||||
requireAddonRuntime()
|
||||
c := base()
|
||||
Expect(os.MkdirAll(c.Root, 0o700)).To(Succeed())
|
||||
store, e := policy.OpenStore(policyDir(c), policy.ModeNamedUser)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
allow := func(context.Context, contract.Actor) error { return nil }
|
||||
p := policy.New(store, policy.Options{Revalidate: allow, CreateAuthority: func(context.Context, contract.Actor, string) error { return nil }})
|
||||
db := authDB()
|
||||
actor := namedActor(db)
|
||||
kept, e := p.Register(context.Background(), actor, "sample")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(p.Commit(context.Background(), actor, kept)).To(Succeed())
|
||||
pending, e := p.Register(context.Background(), actor, "sample")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(store.Close()).To(Succeed())
|
||||
|
||||
a := setup()
|
||||
a.authDB = db
|
||||
Expect(start(a, c)).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
// Committed work stays; the host default drops a registration whose
|
||||
// backing resource nobody can confirm.
|
||||
Expect(a.AddonPolicy().Authorize(context.Background(), actor, kept, policy.ActionRead)).To(Succeed())
|
||||
Expect(a.AddonPolicy().Commit(context.Background(), actor, pending)).NotTo(Succeed())
|
||||
Expect(a.applicationConfig.Addons.OwnershipMode).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("starts with a pending registration whose addon cannot answer and settles it when the addon is ready", func() {
|
||||
requireAddonRuntime()
|
||||
c := base()
|
||||
Expect(os.MkdirAll(c.Root, 0o700)).To(Succeed())
|
||||
store, e := policy.OpenStore(policyDir(c), policy.ModeNamedUser)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
allow := func(context.Context, contract.Actor) error { return nil }
|
||||
p := policy.New(store, policy.Options{Revalidate: allow, CreateAuthority: func(context.Context, contract.Actor, string) error { return nil }})
|
||||
db := authDB()
|
||||
actor := namedActor(db)
|
||||
kept, e := p.Register(context.Background(), actor, "sample")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(p.Commit(context.Background(), actor, kept)).To(Succeed())
|
||||
pending, e := p.Register(context.Background(), actor, "tables")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(store.Close()).To(Succeed())
|
||||
|
||||
var ready atomic.Bool
|
||||
ticks := make(chan time.Time)
|
||||
a := setup()
|
||||
a.authDB = db
|
||||
a.addonReconcileTick = ticks
|
||||
a.addonResourceExists = func(context.Context, policy.ResourceKey) (bool, error) {
|
||||
if !ready.Load() {
|
||||
return false, policy.ErrUnavailable
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
// A disabled or broken addon makes the check unavailable: the host
|
||||
// still starts, the committed resource stays and the record is kept.
|
||||
Expect(start(a, c)).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
Expect(a.AddonPolicy().Authorize(context.Background(), actor, kept, policy.ActionRead)).To(Succeed())
|
||||
ids, _ := a.AddonPolicy().Pending()
|
||||
Expect(ids).To(Equal([]string{pending.ResourceID}))
|
||||
Expect(a.AddonPolicy().Authorize(context.Background(), actor, pending, policy.ActionRead)).NotTo(Succeed())
|
||||
// The addon becomes ready: the retry settles the kept record.
|
||||
ready.Store(true)
|
||||
ticks <- time.Time{}
|
||||
Eventually(func() error {
|
||||
return a.AddonPolicy().Authorize(context.Background(), actor, pending, policy.ActionRead)
|
||||
}).Should(Succeed())
|
||||
ids, _ = a.AddonPolicy().Pending()
|
||||
Expect(ids).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("keeps named-user mode by default and selects local-owner only explicitly with a warning", func() {
|
||||
requireAddonRuntime()
|
||||
logs := &logCapture{}
|
||||
xlog.SetLogger(xlog.NewLoggerWithHandler(slog.NewTextHandler(logs, &slog.HandlerOptions{Level: slog.LevelWarn}), xlog.LogLevelWarn))
|
||||
DeferCleanup(func() { xlog.SetLogger(xlog.NewLogger(xlog.LogLevel("info"), "text")) })
|
||||
|
||||
named := setup()
|
||||
named.authDB = authDB()
|
||||
Expect(start(named, base())).To(Succeed())
|
||||
DeferCleanup(named.Shutdown)
|
||||
Expect(named.AddonOwnershipMode()).To(Equal("named-user"))
|
||||
Expect(logs.String()).NotTo(ContainSubstring("local-owner"))
|
||||
// Named mode still refuses to run without the account database.
|
||||
noDB := setup()
|
||||
Expect(start(noDB, base())).NotTo(Succeed())
|
||||
|
||||
c := base()
|
||||
c.OwnershipMode = "local-owner"
|
||||
a := setup() // no account database: local-owner mode does not need one
|
||||
Expect(start(a, c)).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
Expect(a.AddonOwnershipMode()).To(Equal("local-owner"))
|
||||
Expect(logs.String()).To(ContainSubstring("local-owner"))
|
||||
Expect(logs.String()).To(ContainSubstring("unauthenticated callers share"))
|
||||
Expect(logs.String()).To(ContainSubstring("can reach this port can confirm addon proposals"))
|
||||
local := contract.Actor{UserID: "local-owner", CredentialID: "local-owner", Mode: "local-owner"}
|
||||
key, e := a.AddonPolicy().Register(context.Background(), local, "ghost")
|
||||
Expect(e).To(HaveOccurred(), "creation authority still needs an installed addon")
|
||||
_ = key
|
||||
// Sharing is disabled by the store mode.
|
||||
Expect(a.AddonPolicy().SetGrant(context.Background(), local, policy.ResourceKey{AddonID: "sample", ResourceID: "x"}, "bob", policy.RoleViewer)).NotTo(Succeed())
|
||||
|
||||
bad := base()
|
||||
bad.OwnershipMode = "shared"
|
||||
Expect(start(setup(), bad)).NotTo(Succeed())
|
||||
})
|
||||
})
|
||||
|
||||
type statusFixture struct {
|
||||
s addons.Status
|
||||
err error
|
||||
}
|
||||
|
||||
func (f *statusFixture) Status(context.Context, string) (addons.Status, error) { return f.s, f.err }
|
||||
|
||||
var _ = Describe("addon owner in a full application start", func() {
|
||||
It("starts the named-user gateway when authentication is enabled", func() {
|
||||
requireAddonRuntime()
|
||||
dir := realTempDir()
|
||||
models, backends := filepath.Join(dir, "models"), filepath.Join(dir, "backends")
|
||||
Expect(os.Mkdir(models, 0o750)).To(Succeed())
|
||||
Expect(os.Mkdir(backends, 0o750)).To(Succeed())
|
||||
state, e := system.GetSystemState(system.WithBackendPath(backends), system.WithModelPath(models))
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
DeferCleanup(cancel)
|
||||
root := filepath.Join(dir, "addons")
|
||||
DeferCleanup(func() {
|
||||
_ = filepath.Walk(root, func(p string, i os.FileInfo, e error) error {
|
||||
if e == nil && i.IsDir() {
|
||||
_ = os.Chmod(p, 0700)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
})
|
||||
owner := config.AddonConfig{Enabled: true, InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", StorageID: "local", Root: root, ListenAddress: "127.0.0.1:0", Transport: ownerTransport()}
|
||||
a, e := New(
|
||||
config.WithContext(ctx), config.WithSystemState(state), config.WithDataPath(dir),
|
||||
config.WithAuthEnabled(true), config.WithAuthDatabaseURL(filepath.Join(dir, "auth.db")), config.WithAuthAPIKeyHMACSecret("secret"),
|
||||
config.WithAddons(owner),
|
||||
)
|
||||
Expect(e).NotTo(HaveOccurred(), "the account database must exist before the owner starts: named-user mode checks it")
|
||||
DeferCleanup(a.Shutdown)
|
||||
Expect(a.AuthDB()).NotTo(BeNil())
|
||||
Expect(a.AddonGateway()).NotTo(BeNil())
|
||||
Expect(a.AddonManager()).NotTo(BeNil())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,24 @@
|
||||
//go:build !linux
|
||||
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
addonruntime "github.com/mudler/LocalAI/core/services/addons/runtime"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("addon owner where the runtime is unavailable", func() {
|
||||
It("fails closed instead of starting an owner without a process ledger", func() {
|
||||
owner := config.AddonConfig{Enabled: true, InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", StorageID: "local", Root: filepath.Join(realTempDir(), "addons"), AdminCheck: func(context.Context, contract.Actor) error { return nil }}
|
||||
a := &Application{applicationConfig: &config.ApplicationConfig{Context: context.Background(), Addons: owner}}
|
||||
Expect(a.startAddons()).To(MatchError(addonruntime.ErrRuntime))
|
||||
Expect(a.AddonManager()).To(BeNil())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,268 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
package application
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/core/services/addons"
|
||||
"github.com/mudler/LocalAI/core/services/addons/catalog"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
goruntime "runtime"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
// requireAddonRuntime skips a spec that starts an owner. The add-on runtime
|
||||
// exists on Linux only; other platforms fail closed (see addons_other_test.go).
|
||||
func requireAddonRuntime() {
|
||||
if goruntime.GOOS != "linux" {
|
||||
Skip("the add-on runtime exists on Linux only")
|
||||
}
|
||||
}
|
||||
|
||||
// realTempDir returns a new temporary directory whose path has no symbolic
|
||||
// link. The owner pins every component of its root and refuses a link, and a
|
||||
// temporary directory on macOS lies behind one.
|
||||
func realTempDir() string {
|
||||
dir, err := filepath.EvalSymlinks(GinkgoT().TempDir())
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
return dir
|
||||
}
|
||||
|
||||
var _ = Describe("addon bootstrap", func() {
|
||||
setup := func() *Application {
|
||||
return &Application{applicationConfig: &config.ApplicationConfig{Context: context.Background()}}
|
||||
}
|
||||
cfg := func() config.AddonConfig {
|
||||
return config.AddonConfig{Enabled: true, InstanceID: "owner", OwnerID: "owner", OwnerAddress: "https://owner.example", StorageID: "local", Root: filepath.Join(realTempDir(), "addons"), AdminCheck: func(context.Context, contract.Actor) error { return nil }}
|
||||
}
|
||||
It("installs from a persisted source through application bootstrap", func() {
|
||||
requireAddonRuntime()
|
||||
c := cfg()
|
||||
DeferCleanup(func() {
|
||||
_ = filepath.Walk(c.Root, func(p string, i os.FileInfo, e error) error {
|
||||
if e == nil && i.IsDir() {
|
||||
_ = os.Chmod(p, 0700)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
})
|
||||
c.Host = contract.HostCompatibility{Host: 1, API: 1, Protocol: 1, SDK: 1, DataFormat: 1, OS: "linux", Arch: "amd64"}
|
||||
manifest, e := os.ReadFile("../../tests/addons/fixtures/packages/manifest.json")
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
var b bytes.Buffer
|
||||
tw := tar.NewWriter(&b)
|
||||
for name, data := range map[string][]byte{"manifest.json": manifest, "bin/sample": []byte("binary")} {
|
||||
Expect(tw.WriteHeader(&tar.Header{Name: name, Mode: 0755, Size: int64(len(data))})).To(Succeed())
|
||||
_, e = tw.Write(data)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
}
|
||||
Expect(tw.Close()).To(Succeed())
|
||||
var url string
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/package.tar" {
|
||||
_, _ = w.Write(b.Bytes())
|
||||
return
|
||||
}
|
||||
_ = catalog.WriteIndex(w, []catalog.Release{{ID: "sample", Name: "Sample", Version: "1.2.3", OS: "linux", Arch: "amd64", Artifact: url + "/package.tar", Digest: fmt.Sprintf("sha256:%x", sha256.Sum256(b.Bytes()))}})
|
||||
}))
|
||||
defer server.Close()
|
||||
url = server.URL
|
||||
Expect(os.MkdirAll(c.Root, 0700)).To(Succeed())
|
||||
sources, e := catalog.NewSources(filepath.Join(c.Root, "sources.json"), c.AdminCheck, catalog.CredentialResolver{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, catalog.Source{ID: "test", Name: "Test", URL: url, Enabled: true, AllowLoopbackHTTP: true})).To(Succeed())
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
_, e = a.AddonManager().Submit(context.Background(), contract.Actor{}, addons.LifecycleCommand{OperationID: "install", AddonID: "sample", Action: "install", SourceID: "test", Version: "1.2.3"})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
})
|
||||
|
||||
It("registers the official library once and respects the admin afterwards", func() {
|
||||
requireAddonRuntime()
|
||||
c := cfg()
|
||||
c.OfficialLibrary = true
|
||||
Expect(os.MkdirAll(c.Root, 0700)).To(Succeed())
|
||||
// The default library is on the network; keep this spec offline by
|
||||
// cancelling the startup refresh. The seed happens before it.
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
a.applicationConfig.Context = ctx
|
||||
_ = a.startAddons()
|
||||
sources, e := catalog.NewSources(filepath.Join(c.Root, "sources.json"), c.AdminCheck, catalog.CredentialResolver{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
list, e := sources.List(context.Background(), contract.Actor{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(list).To(HaveLen(1))
|
||||
Expect(list[0].URL).To(Equal(catalog.OfficialLibraryURL))
|
||||
// The admin removes it; the next start does not add it again.
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, catalog.Source{ID: catalog.OfficialLibraryID, Removed: true})).To(Succeed())
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
list, e = a.AddonSources().List(context.Background(), contract.Actor{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(list).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("bounds offline refresh and ignores disabled sources and forwarders", func() {
|
||||
requireAddonRuntime()
|
||||
var calls atomic.Int32
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { calls.Add(1); <-r.Context().Done() }))
|
||||
defer server.Close()
|
||||
c := cfg()
|
||||
Expect(os.MkdirAll(c.Root, 0700)).To(Succeed())
|
||||
sources, e := catalog.NewSources(filepath.Join(c.Root, "sources.json"), c.AdminCheck, catalog.CredentialResolver{})
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
src := catalog.Source{ID: "offline", Name: "Offline", URL: server.URL, Enabled: false}
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, src)).To(Succeed())
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(calls.Load()).To(BeZero())
|
||||
Expect(a.AddonSources()).NotTo(BeNil())
|
||||
_, e = a.AddonSources().Refresh(context.Background(), src.ID)
|
||||
Expect(e).To(MatchError(catalog.ErrUnavailable))
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
src.Enabled = true
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, src)).To(Succeed())
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
a.applicationConfig.Context = ctx
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
Expect(a.AddonManager()).To(BeNil())
|
||||
Expect(a.AddonSources()).To(BeNil())
|
||||
Expect(calls.Load()).To(Equal(int32(1)))
|
||||
c.InstanceID = "frontend"
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(a.AddonSources()).To(BeNil())
|
||||
Expect(calls.Load()).To(Equal(int32(1)))
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
src.Enabled = false
|
||||
Expect(sources.Save(context.Background(), contract.Actor{}, src)).To(Succeed())
|
||||
c.InstanceID = c.OwnerID
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
})
|
||||
It("rejects package-root links during bootstrap without external mutation", func() {
|
||||
c := cfg()
|
||||
Expect(os.MkdirAll(c.Root, 0700)).To(Succeed())
|
||||
external := GinkgoT().TempDir()
|
||||
stage := filepath.Join(external, "stage-keep")
|
||||
Expect(os.Mkdir(stage, 0700)).To(Succeed())
|
||||
DeferCleanup(func() { _ = os.Chmod(stage, 0700) })
|
||||
sentinel := filepath.Join(stage, "keep")
|
||||
Expect(os.WriteFile(sentinel, []byte("keep"), 0400)).To(Succeed())
|
||||
Expect(os.Chmod(stage, 0500)).To(Succeed())
|
||||
Expect(os.Symlink(external, filepath.Join(c.Root, "packages"))).To(Succeed())
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
Expect(a.AddonManager()).To(BeNil())
|
||||
b, e := os.ReadFile(sentinel)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(string(b)).To(Equal("keep"))
|
||||
info, e := os.Stat(stage)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(info.Mode().Perm()).To(Equal(os.FileMode(0500)))
|
||||
})
|
||||
|
||||
It("leaves addons disabled by default without choosing an owner", func() { a := setup(); Expect(a.startAddons()).To(Succeed()); Expect(a.AddonManager()).To(BeNil()) })
|
||||
It("keeps forwarding bootstrap away from storage and rejects absent owners", func() {
|
||||
a := setup()
|
||||
c := cfg()
|
||||
c.InstanceID = "frontend"
|
||||
c.AdminCheck = nil
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
DeferCleanup(a.Shutdown)
|
||||
Expect(c.Root).NotTo(BeADirectory())
|
||||
Expect(a.AddonManager().Reconcile(context.Background())).To(MatchError(addons.ErrNotOwner))
|
||||
c.OwnerID = ""
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
})
|
||||
It("locks explicit owner storage and releases it on application shutdown", func() {
|
||||
requireAddonRuntime()
|
||||
c := cfg()
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
b := setup()
|
||||
b.applicationConfig.Addons = c
|
||||
Expect(b.startAddons()).NotTo(Succeed())
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
Expect(b.startAddons()).To(Succeed())
|
||||
Expect(b.Shutdown()).To(Succeed())
|
||||
c.StorageID = "conflict"
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
})
|
||||
It("reconciles interrupted intent before exposing the manager and cleans failed bootstrap", func() {
|
||||
requireAddonRuntime()
|
||||
c := cfg()
|
||||
a := setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
p := filepath.Join(c.Root, "journal.json")
|
||||
Expect(os.WriteFile(p, []byte(`{"Version":1,"Addons":{},"Operations":{"op":{"Command":{"OperationID":"op","AddonID":"sample","Action":"install"},"Status":{"OperationID":"op","AddonID":"sample","Phase":"intent"}}}}`), 0600)).To(Succeed())
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
b, e := os.ReadFile(p)
|
||||
Expect(e).NotTo(HaveOccurred())
|
||||
Expect(string(b)).To(ContainSubstring(`"Phase":"failed"`))
|
||||
Expect(os.WriteFile(p, []byte(`{`), 0600)).To(Succeed())
|
||||
a = setup()
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
Expect(a.AddonManager()).To(BeNil())
|
||||
Expect(os.Remove(p)).To(Succeed())
|
||||
Expect(a.startAddons()).To(Succeed())
|
||||
Expect(a.Shutdown()).To(Succeed())
|
||||
})
|
||||
It("rejects shared storage and missing authoritative administration", func() {
|
||||
for _, shared := range []bool{true, false} {
|
||||
a := setup()
|
||||
c := cfg()
|
||||
c.SharedStorage = shared
|
||||
if !shared {
|
||||
c.AdminCheck = nil
|
||||
}
|
||||
a.applicationConfig.Addons = c
|
||||
Expect(a.startAddons()).NotTo(Succeed())
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("addon gateway bootstrap", func() {
|
||||
It("keeps public gateway unavailable without explicit transport bootstrap", func() {
|
||||
a := &Application{applicationConfig: &config.ApplicationConfig{Context: context.Background()}}
|
||||
Expect(a.AddonGateway()).To(BeNil())
|
||||
})
|
||||
})
|
||||
@@ -3,8 +3,11 @@ package application
|
||||
import (
|
||||
"cmp"
|
||||
"context"
|
||||
"errors"
|
||||
"math/rand/v2"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
@@ -13,6 +16,15 @@ import (
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/core/http/auth"
|
||||
mcpTools "github.com/mudler/LocalAI/core/http/endpoints/mcp"
|
||||
"github.com/mudler/LocalAI/core/services/addons"
|
||||
"github.com/mudler/LocalAI/core/services/addons/catalog"
|
||||
"github.com/mudler/LocalAI/core/services/addons/gateway"
|
||||
"github.com/mudler/LocalAI/core/services/addons/packages"
|
||||
"github.com/mudler/LocalAI/core/services/addons/pins"
|
||||
"github.com/mudler/LocalAI/core/services/addons/policy"
|
||||
"github.com/mudler/LocalAI/core/services/addons/recovery"
|
||||
addonruntime "github.com/mudler/LocalAI/core/services/addons/runtime"
|
||||
"github.com/mudler/LocalAI/core/services/addons/updates"
|
||||
"github.com/mudler/LocalAI/core/services/advisorylock"
|
||||
"github.com/mudler/LocalAI/core/services/agentpool"
|
||||
"github.com/mudler/LocalAI/core/services/chathistory"
|
||||
@@ -33,6 +45,7 @@ import (
|
||||
"github.com/mudler/LocalAI/core/services/voicerecognition"
|
||||
"github.com/mudler/LocalAI/core/templates"
|
||||
"github.com/mudler/LocalAI/core/trace"
|
||||
"github.com/mudler/LocalAI/pkg/addons/contract"
|
||||
pkggrpc "github.com/mudler/LocalAI/pkg/grpc"
|
||||
localaitools "github.com/mudler/LocalAI/pkg/mcp/localaitools"
|
||||
localaiInproc "github.com/mudler/LocalAI/pkg/mcp/localaitools/inproc"
|
||||
@@ -57,6 +70,22 @@ const faceEmbeddingDim = 0
|
||||
const voiceEmbeddingDim = 0
|
||||
|
||||
type Application struct {
|
||||
addonPolicy *policy.Policy
|
||||
addonStore *policy.Store
|
||||
addonResourceExists func(context.Context, policy.ResourceKey) (bool, error)
|
||||
// addonReconcileTick drives the pending-registration retry; nil is a real
|
||||
// 30 second ticker. addonReconcile is that loop once the owner started it.
|
||||
addonReconcileTick <-chan time.Time
|
||||
addonReconcile *policy.ReconcileLoop
|
||||
addonGateway *gateway.Gateway
|
||||
addonServer *http.Server
|
||||
addonSources *catalog.Sources
|
||||
addonManager *addons.Manager
|
||||
addonHostOps *addonruntime.HostOps
|
||||
// The pin store is built once so the routes and the retention sweep share it.
|
||||
addonPinStoreOnce sync.Once
|
||||
addonPinStore pins.Store
|
||||
addonPinStoreErr error
|
||||
webSocketTicketsOnce sync.Once
|
||||
webSocketTickets *auth.WebSocketTickets
|
||||
backendLoader *config.ModelConfigLoader
|
||||
@@ -549,12 +578,21 @@ func (a *Application) IsDistributed() bool {
|
||||
func (a *Application) Shutdown() error {
|
||||
var err error
|
||||
a.shutdownOnce.Do(func() {
|
||||
if a.addonServer != nil {
|
||||
_ = a.addonServer.Close()
|
||||
}
|
||||
if a.addonStore != nil {
|
||||
_ = a.addonStore.Close()
|
||||
}
|
||||
if a.addonManager != nil {
|
||||
err = a.addonManager.Close()
|
||||
}
|
||||
// Before distributed shutdown: the sync's subscriptions live on the
|
||||
// NATS connection that closes there.
|
||||
a.stopFailoverDistributed()
|
||||
a.distributed.Shutdown()
|
||||
if a.modelLoader != nil {
|
||||
err = a.modelLoader.StopAllGRPC()
|
||||
err = errors.Join(err, a.modelLoader.StopAllGRPC())
|
||||
}
|
||||
if a.voiceProfileStore != nil {
|
||||
if closeErr := a.voiceProfileStore.Close(); err == nil {
|
||||
@@ -585,6 +623,7 @@ func (a *Application) waitForHealthyWorker() {
|
||||
}
|
||||
}
|
||||
// Add 0-1s jitter to prevent thundering-herd on the node registry
|
||||
// #nosec G404 -- jitter only spreads retries, it is not security sensitive.
|
||||
jitter := time.Duration(rand.Int64N(int64(time.Second)))
|
||||
select {
|
||||
case <-a.applicationConfig.Context.Done():
|
||||
@@ -600,7 +639,16 @@ func (a *Application) InstanceID() string {
|
||||
return a.applicationConfig.Distributed.InstanceID
|
||||
}
|
||||
|
||||
func (a *Application) start() error {
|
||||
func (a *Application) start() (startErr error) {
|
||||
if err := a.startAddons(); err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() {
|
||||
if startErr != nil && a.addonManager != nil {
|
||||
_ = a.addonManager.Close()
|
||||
}
|
||||
}()
|
||||
|
||||
galleryService := galleryop.NewGalleryService(a.ApplicationConfig(), a.ModelLoader())
|
||||
err := galleryService.Start(a.ApplicationConfig().Context, a.ModelConfigLoader(), a.ApplicationConfig().SystemState)
|
||||
if err != nil {
|
||||
@@ -772,3 +820,129 @@ func (a *Application) StartAgentPool() {
|
||||
|
||||
a.agentPoolService.Store(aps)
|
||||
}
|
||||
|
||||
// AddonManager is nil unless the deployment explicitly enables addons.
|
||||
func (a *Application) AddonManager() *addons.Manager { return a.addonManager }
|
||||
|
||||
// AddonSources exposes owner-only source administration and explicit live refresh.
|
||||
// Forwarders never load source configuration or perform catalog I/O.
|
||||
func (a *Application) AddonSources() *catalog.Sources { return a.addonSources }
|
||||
|
||||
func (a *Application) startAddons() error {
|
||||
c := a.applicationConfig.Addons
|
||||
if !c.Enabled {
|
||||
return nil
|
||||
}
|
||||
if a.addonManager != nil {
|
||||
return addons.ErrConflict
|
||||
}
|
||||
mode := a.AddonOwnershipMode()
|
||||
if mode != auth.AddonModeNamedUser && mode != auth.AddonModeLocalOwner {
|
||||
return addons.ErrConfiguration
|
||||
}
|
||||
if c.Transport != nil {
|
||||
switch mode {
|
||||
case auth.AddonModeLocalOwner:
|
||||
// The HTTP layer checked the caller's role when an account database
|
||||
// exists; here the only principal is the local owner.
|
||||
c.AdminCheck = func(_ context.Context, actor contract.Actor) error { return auth.ValidateLocalOwnerActor(actor) }
|
||||
default:
|
||||
if a.authDB == nil {
|
||||
return addons.ErrForbidden
|
||||
}
|
||||
c.AdminCheck = func(ctx context.Context, actor contract.Actor) error {
|
||||
return auth.ValidateAddonActor(ctx, a.authDB, actor, true)
|
||||
}
|
||||
}
|
||||
}
|
||||
owner := addons.OwnerConfig{InstanceID: c.InstanceID, OwnerID: c.OwnerID, OwnerAddress: c.OwnerAddress, StorageID: c.StorageID, Root: c.Root, SharedStorage: c.SharedStorage}
|
||||
var cat catalog.Catalog
|
||||
var installer *packages.Installer
|
||||
// Replicas must not load even source metadata from the owner's volume.
|
||||
if c.InstanceID == c.OwnerID && c.OwnerID != "" {
|
||||
if c.AdminCheck == nil {
|
||||
return addons.ErrForbidden
|
||||
}
|
||||
sources, err := catalog.NewSources(filepath.Join(c.Root, "sources.json"), c.AdminCheck, catalog.CredentialResolver{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if c.OfficialLibrary {
|
||||
if _, err = sources.SeedDefault(catalog.OfficialLibrary()); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
host := c.Host
|
||||
if host == (contract.HostCompatibility{}) {
|
||||
host = contract.HostCompatibility{Host: 1, API: 1, Protocol: 1, SDK: contract.HostSDKVersion, DataFormat: 1, OS: runtime.GOOS, Arch: runtime.GOARCH}
|
||||
}
|
||||
cat = catalog.Catalog{Sources: sources, TargetOS: host.OS, TargetArch: host.Arch}
|
||||
installer = &packages.Installer{Root: filepath.Join(c.Root, "packages"), Catalog: cat, Host: host}
|
||||
}
|
||||
m, err := addons.NewManager(a.applicationConfig.Context, owner, cat, installer, c.AdminCheck)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if c.InstanceID == c.OwnerID {
|
||||
// Data recovery sits beside the manager: an upgrade of an add-on that
|
||||
// has data goes through it, and its journals are settled before any
|
||||
// process starts. It gets the owner's policy once that exists.
|
||||
account := func(context.Context, string) error { return gateway.ErrForbidden }
|
||||
if a.AddonOwnershipMode() != auth.AddonModeLocalOwner && a.authDB != nil {
|
||||
users := auth.NewAddonUsers(a.authDB)
|
||||
account = users.Active
|
||||
}
|
||||
rec, rerr := recovery.New(recovery.Config{
|
||||
Root: c.Root, Host: m.RecoveryHost(), Admin: c.AdminCheck, Account: account,
|
||||
SelectionsDir: filepath.Join(c.Root, "selections"),
|
||||
Retention: recovery.Retention{Keep: c.BackupKeep, MaxAge: c.BackupMaxAge},
|
||||
})
|
||||
if rerr != nil {
|
||||
_ = m.Close()
|
||||
return rerr
|
||||
}
|
||||
m.SetRecovery(rec)
|
||||
// The update check reads the libraries and the journal the owner
|
||||
// holds; it applies nothing, and its cache is bounded.
|
||||
m.EnableUpdates(updates.Config{TTL: c.UpdateCacheTTL, MinRefresh: c.UpdateRefreshInterval})
|
||||
if err = m.Reconcile(a.applicationConfig.Context); err != nil {
|
||||
_ = m.Close()
|
||||
return err
|
||||
}
|
||||
}
|
||||
if cat.Sources != nil {
|
||||
if err = cat.Sources.RefreshEnabled(a.applicationConfig.Context); err != nil {
|
||||
if a.applicationConfig.Context.Err() != nil {
|
||||
_ = m.Close()
|
||||
return err
|
||||
}
|
||||
// An unreachable library must not keep LocalAI or the installed
|
||||
// add-ons down. The cache stays empty until an explicit refresh.
|
||||
xlog.Warn("Could not refresh the add-on libraries at startup", "error", err)
|
||||
}
|
||||
}
|
||||
if c.InstanceID == c.OwnerID {
|
||||
limits := contract.DefaultLimits()
|
||||
a.addonHostOps = a.newAddonHostOps(m, limits)
|
||||
supervisor, err := addonruntime.NewSupervisor(addonruntime.Config{StateRoot: filepath.Join(c.Root, "runtime"), DataRoot: filepath.Join(c.Root, "data"), Limits: limits, ReadyTimeout: limits.ReadTimeout, StableHealth: limits.BackoffMax, HostOps: a.addonHostOps})
|
||||
if err != nil {
|
||||
_ = m.Close()
|
||||
return err
|
||||
}
|
||||
if err = m.AttachRuntime(a.applicationConfig.Context, supervisor); err != nil {
|
||||
_ = supervisor.Close(context.Background())
|
||||
_ = m.Close()
|
||||
return err
|
||||
}
|
||||
}
|
||||
a.addonSources = cat.Sources
|
||||
a.addonManager = m
|
||||
if c.InstanceID == c.OwnerID {
|
||||
go a.runAddonAutoUpgrade(a.applicationConfig.Context, upgradeInitialDelay, upgradeCheckInterval)
|
||||
}
|
||||
if err := a.startAddonGateway(); err != nil {
|
||||
_ = m.Close()
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -68,6 +68,12 @@ var _ = Describe("loadRuntimeSettingsFromFile", func() {
|
||||
Expect(cfg.AutoUpgradeBackends).To(BeTrue())
|
||||
})
|
||||
|
||||
It("loads auto_upgrade_addons", func() {
|
||||
cfg := &config.ApplicationConfig{DynamicConfigsDir: seedSettings(`{"auto_upgrade_addons": true}`)}
|
||||
loadRuntimeSettingsFromFile(cfg)
|
||||
Expect(cfg.AutoUpgradeAddons).To(BeTrue())
|
||||
})
|
||||
|
||||
It("loads prefer_development_backends", func() {
|
||||
cfg := &config.ApplicationConfig{DynamicConfigsDir: seedSettings(`{"prefer_development_backends": true}`)}
|
||||
loadRuntimeSettingsFromFile(cfg)
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/core/backend"
|
||||
@@ -69,7 +70,26 @@ func New(opts ...config.AppOption) (*Application, error) {
|
||||
xlog.Info("Starting LocalAI", "threads", options.Threads, "modelsPath", options.SystemState.Model.ModelsPath)
|
||||
xlog.Info("LocalAI version", "version", internal.PrintableVersion())
|
||||
|
||||
// The account database is opened before the services start: the add-on
|
||||
// owner checks its callers against it in named-user mode, and has no
|
||||
// identity to check them with when it starts first.
|
||||
if options.Auth.Enabled {
|
||||
if err := ensureAuthDBDirs(options); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
authDB, err := auth.InitDB(options.Auth.DatabaseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to initialize auth database: %w", err)
|
||||
}
|
||||
application.authDB = authDB
|
||||
}
|
||||
|
||||
if err := application.start(); err != nil {
|
||||
if application.authDB != nil {
|
||||
if sqlDB, e := application.authDB.DB(); e == nil {
|
||||
_ = sqlDB.Close()
|
||||
}
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -148,11 +168,7 @@ func New(opts ...config.AppOption) (*Application, error) {
|
||||
options.Auth.APIKeyHMACSecret = secret
|
||||
}
|
||||
|
||||
authDB, err := auth.InitDB(options.Auth.DatabaseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to initialize auth database: %w", err)
|
||||
}
|
||||
application.authDB = authDB
|
||||
authDB := application.authDB
|
||||
xlog.Info("Auth enabled", "database", sanitize.URL(options.Auth.DatabaseURL))
|
||||
|
||||
// Start session and expired API key cleanup goroutine
|
||||
@@ -714,7 +730,7 @@ func initializeWatchdog(application *Application, options *config.ApplicationCon
|
||||
// loadOrGenerateHMACSecret loads an HMAC secret from the given file path,
|
||||
// or generates a random 32-byte secret and persists it if the file doesn't exist.
|
||||
func loadOrGenerateHMACSecret(path string) (string, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
data, err := os.ReadFile(filepath.Clean(path))
|
||||
if err == nil {
|
||||
secret := string(data)
|
||||
if len(secret) >= 32 {
|
||||
@@ -792,3 +808,33 @@ func loadStartupSettings(options *config.ApplicationConfig) {
|
||||
loadRuntimeSettingsFromFile(options)
|
||||
options.SystemState.PreferDevelopmentBackends = options.PreferDevelopmentBackends || options.SystemState.PreferDevelopmentBackends
|
||||
}
|
||||
|
||||
// ensureAuthDBDirs creates the directories the auth database file lives in.
|
||||
// The database opens before the general directory setup, and its default
|
||||
// location is inside the data directory, which may not exist on first start.
|
||||
// Directories keep the 0750 mode of the later setup. PostgreSQL URLs and
|
||||
// in-memory databases have no directory.
|
||||
func ensureAuthDBDirs(options *config.ApplicationConfig) error {
|
||||
if options.DataPath != "" {
|
||||
if err := os.MkdirAll(options.DataPath, 0o750); err != nil {
|
||||
return fmt.Errorf("unable to create DataPath: %q", err)
|
||||
}
|
||||
}
|
||||
u := options.Auth.DatabaseURL
|
||||
if strings.HasPrefix(u, "postgres://") || strings.HasPrefix(u, "postgresql://") {
|
||||
return nil
|
||||
}
|
||||
if i := strings.IndexByte(u, '?'); i >= 0 {
|
||||
u = u[:i]
|
||||
}
|
||||
u = strings.TrimPrefix(u, "file:")
|
||||
if u == "" || u == ":memory:" || strings.HasPrefix(u, ":memory:") {
|
||||
return nil
|
||||
}
|
||||
if dir := filepath.Dir(u); dir != "." {
|
||||
if err := os.MkdirAll(dir, 0o750); err != nil {
|
||||
return fmt.Errorf("unable to create the auth database directory: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
//go:build auth
|
||||
|
||||
package application
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/mudler/LocalAI/core/config"
|
||||
"github.com/mudler/LocalAI/pkg/system"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("auth database location at first start", func() {
|
||||
It("creates a missing data directory before opening the default auth database", func() {
|
||||
root := GinkgoT().TempDir()
|
||||
models, backends := filepath.Join(root, "models"), filepath.Join(root, "backends")
|
||||
Expect(os.Mkdir(models, 0750)).To(Succeed())
|
||||
Expect(os.Mkdir(backends, 0750)).To(Succeed())
|
||||
state, err := system.GetSystemState(system.WithBackendPath(backends), system.WithModelPath(models))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
data := filepath.Join(root, "not", "yet", "created")
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
var app *Application
|
||||
app, err = New(config.WithContext(ctx), config.WithSystemState(state), config.WithDataPath(data),
|
||||
config.WithAuthEnabled(true), config.WithAuthDatabaseURL(filepath.Join(data, "database.db")),
|
||||
config.WithAuthAPIKeyHMACSecret("secret"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
DeferCleanup(func() { cancel(); _ = app.Shutdown() })
|
||||
Expect(app.AuthDB()).NotTo(BeNil())
|
||||
info, err := os.Stat(data)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(info.Mode().Perm()&0o027).To(BeZero(), "directory keeps the 0750 semantics")
|
||||
_, err = os.Stat(filepath.Join(data, "database.db"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
})
|
||||
It("creates the parent of an explicit sqlite path outside the data directory", func() {
|
||||
root := GinkgoT().TempDir()
|
||||
models, backends := filepath.Join(root, "models"), filepath.Join(root, "backends")
|
||||
Expect(os.Mkdir(models, 0750)).To(Succeed())
|
||||
Expect(os.Mkdir(backends, 0750)).To(Succeed())
|
||||
state, err := system.GetSystemState(system.WithBackendPath(backends), system.WithModelPath(models))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
var app *Application
|
||||
db := filepath.Join(root, "elsewhere", "auth", "users.db?_busy_timeout=2000")
|
||||
app, err = New(config.WithContext(ctx), config.WithSystemState(state), config.WithDataPath(filepath.Join(root, "data")),
|
||||
config.WithAuthEnabled(true), config.WithAuthDatabaseURL(db), config.WithAuthAPIKeyHMACSecret("secret"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
DeferCleanup(func() { cancel(); _ = app.Shutdown() })
|
||||
Expect(app.AuthDB()).NotTo(BeNil())
|
||||
_, err = os.Stat(filepath.Join(root, "elsewhere", "auth", "users.db"))
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
})
|
||||
})
|
||||
@@ -15,6 +15,13 @@ import (
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// The schedule of the periodic upgrade checks. The backend check and the
|
||||
// add-on check both follow it.
|
||||
const (
|
||||
upgradeInitialDelay = 10 * time.Second
|
||||
upgradeCheckInterval = 6 * time.Hour
|
||||
)
|
||||
|
||||
// UpgradeChecker periodically checks for backend upgrades and optionally
|
||||
// auto-upgrades them. It caches the last check results for API queries.
|
||||
//
|
||||
@@ -58,7 +65,7 @@ func NewUpgradeChecker(appConfig *config.ApplicationConfig, ml *model.ModelLoade
|
||||
systemState: appConfig.SystemState,
|
||||
db: db,
|
||||
backendManagerFn: backendManagerFn,
|
||||
checkInterval: 6 * time.Hour,
|
||||
checkInterval: upgradeCheckInterval,
|
||||
stop: make(chan struct{}),
|
||||
done: make(chan struct{}),
|
||||
triggerCh: make(chan struct{}, 1),
|
||||
@@ -78,7 +85,7 @@ func (uc *UpgradeChecker) Run(ctx context.Context) {
|
||||
// Initial delay: don't slow down startup. Short enough that operators
|
||||
// don't stare at an empty upgrade banner for long; long enough that
|
||||
// workers have registered and reported their installed backends.
|
||||
initialDelay := 10 * time.Second
|
||||
initialDelay := upgradeInitialDelay
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
|
||||
Reference in new issue
Block a user