From e45dc5e9f72ea510bc8aab3248d2bef3be8f2263 Mon Sep 17 00:00:00 2001 From: mudler-agent Date: Thu, 22 Oct 2026 17:57:19 +0200 Subject: [PATCH] feat(addons): installable add-ons foundation and the Tables add-on (#12702) * feat(addons): add the add-on runtime, navigation and first-party add-ons Add the add-on runtime to LocalAI: manifests, the host operation broker, admin gating by account role, the add-on UI bundle builder and the SDK for add-on pages. Move the navigation to Chat, Models, Add-ons and Settings, with pinned add-ons in the sidebar and an Add-ons drawer. Ship the first-party add-ons (voice, face, fine-tuning, quantization) as packages for linux/amd64, linux/arm64 and darwin/arm64. Intel macOS (darwin/amd64) is enabled in the platform files. Squashed from the review branch. The tree is unchanged; the pre-squash history is kept on a local backup branch. Signed-off-by: Ettore Di Giacinto * fix(gosec): tighten file modes and annotate false positives Write job state, configs and imported models with mode 0600, clean the paths that are read from disk, and mark the route tables, the header name and the retry jitter as reviewed false positives. Assisted-by: Claude Code:claude-sonnet-5-5 gosec Signed-off-by: Ettore Di Giacinto * fix(addons): keep API keys off recovery mutations and same-site cookies POST /api/addons/dispatch forwarded the recovery and update mutations to the owner for any administrator key, which skipped the browser-session rule of their own routes. Refuse those actions there. Also apply the same-origin rule to the legacy token cookie, not only the session cookie, because the global CSRF check lets same-site requests through. Do not fail the start of LocalAI when an add-on library cannot be refreshed; log it and keep the installed add-ons up. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(addons): refuse a reinstall that would migrate retained data unbacked An upgrade takes a verified backup before the new package migrates the data. A reinstall over retained data has no running generation to stop, so the new package would migrate it without one. Refuse a reinstall whose data format differs from the retained one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(addons): release the manager lock while a release is fetched Submit held the state lock across the catalog lookup and the package download, so the apps list and status reads stalled for the whole install. Serialize Submit and Reconcile with their own lock and drop the state lock around the fetch only. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(addons): kill an add-on only after three missed health probes One slow 100 ms probe killed a busy add-on and dropped the work it was serving. Require three misses in a row. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(ui): move settings under the operate hub The Operate hub already has Settings as a tab, so the sidebar no longer shows a separate Settings entry. The /app/settings route stays, and the hub tab still reaches it. The navigation test now expects Chat, Models and Add-ons as the primary items. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * fix(addons): accept same-host browser requests that carry no fetch metadata Browsers do not send Sec-Fetch-* headers over plain HTTP on an address that is not localhost, so every add-on request from a LAN address was refused as cross-site. Keep refusing a Sec-Fetch-Site value other than same-origin. When the header is absent, accept the request only if its Origin, or failing that its Referer, names the same host and port as the request. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(ui): show the server's reason when an add-on request is refused A 403 or 400 answer showed a generic message and hid the reason in the response body, so a refusal by the browser guard could not be told from a missing permission. Show error.message from the body as the server sent it, and keep the generic text when the body has none. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * feat(addons): run the add-on owner in the normal binary The owner was only enabled when an embedding host set the add-on config, so a normal run answered every add-on request as unavailable. Add --addons-enabled (default true) and --addons-ownership-mode (default named-user, or local-owner; anything else stops startup). The node is its own owner: state lives under the data path, and the carrier uses loopback keys made at each start and kept in memory. --addons-enabled=false turns the owner off completely. Named-user mode needs the account database. With authentication off the owner stays off and a warning says how to use local-owner. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(addons): keep the loopback owner address stable across restarts The single-node loopback transport took a new ephemeral port at each start. The owner stores its address in its identity file and refuses to start when the address differs, so every start after the first failed with "invalid addon owner configuration". Choose the port once, keep it in the add-on state root (loopback-port, mode 0600) and reuse it. If the port is taken at a later start, stop with an error that names the port and --addons-enabled=false, because another port would break the identity check. The keys and certificate stay in memory. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(addons): let add-on pages keep the host modules they were written against A page written for the web UI imports React, the router, i18next and a few shared components. The add-on builder refuses a bundle that holds a host file, so moving such a page into an add-on meant rewriting it. Add a compat mode to the shared builder. It answers those imports from a directory of stand-ins that read the SDK, bundles the dynamic import of the page code into the one output file, and collects the style sheets the page imports. A relative import that is not in the add-on's own source resolves to the file at the same path under the compat directory. Widen the SDK for these pages, without a version change: more host components (spinner, modal, lightbox, model picker) and hooks (models, resources, voice profiles, operations, MCP client), a style on icons, the icon names the pages draw, sdk.navigation.openApp to open a page of the app (only addresses under /app, and the way back as the only state) and sdk.portal. The shell passes the query string to the page. The stand-ins cover the router (the tab in the path, the model in the query), the translator over the host locale files, the core API calls through sdk.core.fetch, and byte copies of the pure helpers, with tests that keep the copies equal to the originals. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * feat(addons): ship Studio and Talk as add-ons instead of built-in pages Studio and Talk were compiled into the web bundle and always listed in the Add-ons drawer. They are add-ons now, like Voice, Face, fine-tuning and quantization: a person installs and removes them, the drawer lists them only while they are installed, and they are packaged the same way. Move the pages, their components, hooks and helpers into addons/studio and addons/talk. Each has a manifest, a UI bundle, a release platform matrix and a catalog template, and is built by the compat mode of the shared builder. The core API stays where it is: the pages call it with the signed-in person's session. The history keeps its browser storage keys. The unit tests of the moved helpers move with them. In the core, drop the two pages, the Create group of the sidebar (Chat joins Home and Models in the old layout), the built-in catalog entries (the catalog is empty) and the talk locale namespace. The old addresses (/app/studio, /app/talk and the generator links) redirect to the add-on pages, with the model as ?model=. The Home /studio command is offered only while the Studio add-on is installed and opens it. The release workflow and its tests now list seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto * test(ui): run the Studio and Talk specs against the add-on bundles The pages are add-ons now, so a spec that draws one installs the real built bundle first: the apps list names it and its module and style sheet are served from the build output, the way the add-on owner serves them. studio-addon.js holds that helper, and the Studio and Talk fixtures call it. Update the specs for the new addresses (/app/addons/studio/studio, the model as ?model=), the session permission the add-on route needs, the legacy redirects, the Home /studio command that appears only while the add-on is installed, and the sidebar without a Create group. Add specs for the add-ons themselves: every tab draws with no script error, the shell titles the page once, the old addresses say so when the add-on is not installed, a button of the page reaches a core page with the way back, and the drawer lists both only after they are installed. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * fix(ui): centre the icon of a built-in tile in its icon box The drawer drew the icon of a built-in tile (Studio's palette was one) in a bare 40px box. Only the placeholder box was laid out as a centred grid with the 20px size, so the real icon sat in the top-left corner at the size of the page text. Give the built-in tile the same classes as the placeholder box, and check in a browser that the icon is centred. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * docs(addons): describe Studio and Talk as add-ons Replace the built-in list (now empty) with the Studio and Talk add-on sections, the new targets of the old links, the Home command, and the seven packaged add-ons. Describe the SDK additions the pages use: components, hooks, icons, openApp and portal. Say on the Studio page that it is installed from the drawer. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * test(ui): click the shell title in the Studio slash-key spec The add-on shell titles the page, and the Studio overview no longer draws a second title, but the generator tabs keep theirs, so a bare h1 locator matched two elements. Click the first, which is the shell's. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * fix(addons): read the loopback port file through os.Root The persisted port file was read with a path built from the add-on root, which gosec reports as a file inclusion through a variable. Open it through os.Root so the name cannot leave the root, even by a symbolic link, and require a regular file with no group or other access. A looser file now fails closed instead of being trusted. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * test(ui): expect Settings under the Operate hub, not the sidebar Settings moved out of the sidebar into the Operate hub. The navigation spec still expected a sidebar link. Assert that the sidebar has no Settings entry, and that the Operate hub opens Settings as a tab. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(ui): drop Settings from the 404 places Settings is a tab of the Operate hub and not a destination of its own, so the 404 page does not list it. Assert that it is absent. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(ui): drive the media history and TTS specs through the Studio add-on The image, video, sound and speech generators are pages of the Studio add-on now, and the old addresses redirect to it. Install the add-on with the Studio fixtures, open its pages and use its compose card, so the specs run against the page that exists. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(ui): count the first-run steps, not every list item The recommended models are a list inside the second step, so counting all list items read 3 only while the models were still loading. Count the step elements, which does not depend on the load order. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(ui): fail the node refresh by state, not by call count The spec served the list to the first request and failed the rest. The first request is not always the one from the page, so the page could get the failure first and show no list. Serve the list until the page shows it, then fail the refreshes. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * feat(addons): default to local-owner when auth is off A server without authentication has no accounts, so named-user left the add-on owner off. Resolve the unset mode in one place: local-owner when authentication is off, named-user when it is on. A flag or environment value still wins, and an unknown value is still refused. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(addons): register an official add-on library by default Add gallery/addons/index.yaml, a YAML index of the four first-party add-ons (fine-tuning, quantization, voice, face) in the library format. The entries come from addonpkg catalog over locally built linux/amd64 packages, so the digests pin those images. On the first start, when the owner has no sources.json, register the raw GitHub URL of that index on master as the "official" library and write the file. The file is then the record: a library the admin removes stays removed, and an existing file is never changed. --addons-official-library=false skips the default. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto * docs(addons): document the ownership default and the official library Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * feat(addons): list Studio and Talk in the first-party index Add the Studio and Talk entries to gallery/addons/index.yaml next to the four existing ones. They come from addonpkg catalog over locally built linux/amd64 packages, so the digests pin those images. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto * test(addons): expect Studio and Talk in the official index The official index now lists six add-ons. Update the test that decodes it so it accepts Studio and Talk. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(addons): add a real icon to each first-party add-on Each icon is a 256 by 256 PNG: a rounded square in one colour of the UI kit palette with a white outline glyph from the kit sprite. Fine-tuning uses sliders, quantization layers, voice waveform, face user, studio palette, talk phone and tables database. They were rendered with headless Chrome from the sprite paths. Assisted-by: Claude Code:claude-sonnet-5-5 chrome Signed-off-by: Ettore Di Giacinto * feat(addons): ship a committed icon.png instead of the generated square The shared UI builder and the Tables builder copy ui/icon.png from the add-on source tree to the package unchanged when the file exists. They write the generated placeholder only when it does not. Add a fixture with an icon and a test for both cases. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * feat(addons): list the tables add-on in the first-party index Add the Tables entry to gallery/addons/index.yaml next to the six existing ones. It comes from addonpkg catalog over a locally built linux/amd64 package, so the digest pins that image. The official index test now expects seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto * feat(addons): make the Voice add-on one page with tabs The Voice add-on had four pages: recognition, library, create and speech, and the drawer listed two of them. It now has one page, "voice", and one drawer entry. Recognition, Library, Text to speech and From a recording are tabs of that page, drawn with the same tab strip the recognition page already used. The tab is the first path segment (voice/library), as in the Studio add-on, and the form that adds a voice is a view of the Library tab (voice/library/new). The old page names stay declared in the manifest. Each one redirects to its tab and keeps the query string, so saved links still open. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * feat(ui): send the old voice routes to the Voice tabs /app/voice, /app/voice-library and /app/voice-library/new now open the matching tab of the Voice add-on page while the add-on is installed and running, and the core pages otherwise. The moved-page helper compares the first segment of the target with the declared pages, so a target can name a tab. The docs and the navigation spec list the new addresses. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(addons): cover the Voice tabs and the old page redirects The add-on unit tests render the tabs instead of the four pages. The Playwright spec checks the tab strip, the address of each tab, the missing Library tab for a member, and that the old page addresses of the add-on redirect to their tab with the query string kept. The migration table test follows the new targets. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * fix(studio): keep the model menu inside a narrow screen The Home model picker sizes its menu from the viewport, and assumes the picker sits at the page gutter. In the Studio composer card the field is further in, so on a 390px screen the menu of the Sound expansion picker ended 7px past the right edge. Size the menu from the field it opens under instead. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): build the add-on bundles once for the whole run Each worker built the Studio and Talk bundles into the same dist directory, and the build empties that directory first. A worker that started, or restarted after a failure, removed the files another worker was serving, so a Studio spec saw "the style sheet of the add-on could not be loaded". Build once in a global setup before the workers start. A run without the setup still builds per worker. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): register the worker when the add-node spec says so The mocked roster added the worker on the third read of /api/nodes. The shell and the page both read it, and which one asks first depends on load, so the page could take the new worker as part of the roster it opened with and never show it as found. The worker now joins after the page shows it is listening. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): hold the gallery answer until the skeleton is seen The spec delayed the answer by a fixed 800 ms and then looked for the skeleton. The assertion polls at growing intervals, so on a slow machine it could step over that window. The answer now waits until the skeleton has been seen, and the spec then checks that the skeleton goes away. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): give the PDF tag the time the other PDF specs have pdf.js and its worker load on first use. The other specs in this file wait up to ten seconds for the result of that load. The first one waited five, and missed the tag on a busy machine. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): build the Face and Fine-tuning bundles in the global setup The navigation spec serves the built Fine-tuning bundle, but nothing built it, so two specs failed on a fresh checkout when the dist folder was missing. The spec also built the Face bundle in a hook that every worker runs, and a build rewrites the folder the other workers serve. The global setup now builds Studio, Talk, Face and Fine-tuning once, before the workers start. The spec builds the two bundles itself only when it runs without the global setup. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): fix the Home answers in the stale pin spec The spec asserts that the page shows no alert. The Home page raises its own alert when the model list fails or a staging operation has failed, and both answers come from the shared test server. A slow reply or a failed install left there counted as an alert about the pin. The spec now fixes both answers, so the alert count tells only about the pin. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(ui): read the server address in the add-node spec from the page The spec expected the register address to carry port 8089, so it failed on any other port. It now expects the origin the page was opened from. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto * test(addons): give the navigation routes spec its own catalog No page is built in today, so the catalog is empty and the three specs that expect the studio and talk entries failed on the list and on the pin seed. They passed only while Studio and Talk were built in. The spec now sets a catalog of two ungated entries for each example and restores the old one afterwards. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(cli): report a failed write of the MLX hostfile The tunnel callback ignored the error of os.WriteFile and set the hostfile variable even when the file was not written. It now logs the error and leaves the variable alone. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(addons): discard the close error of the held listener errcheck flags the deferred Close of the listener that keeps the port busy. The test has no use for that error, so the defer drops it explicitly. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(addons): move the add-on tool tests to Ginkgo The fine-tuning, quantization and add-on host library tests used t.Errorf, t.Fatal and t.Run, which the forbidigo rule rejects. They now use Describe, It and Gomega, with a suite bootstrap in each package. The cases and the assertions are the same. The tables in the host client and the tool schema tests keep one entry per case. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(addons): move the host operation tests to Ginkgo The hostops tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs under one suite bootstrap. Every check and every per-operation case is kept. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(addons): move the pin store tests to Ginkgo The pin service, file store, sweep and shared store contract tests used the testing package calls that forbidigo rejects. The store contract is now a function that registers its specs for each store, so the file store, the memory-only store and the database store run the same cases. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(addons): move the remaining add-on tests to Ginkgo The built-in catalog, tool arguments, loopback carrier, host operation contract, packaging, application, config, CLI and endpoint tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs. The endpoint tests also drop the unchecked Close errors that errcheck reports when the auth tag is on. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(ui): wait for the targeted Reinstall button before the install click The "generic deletion blocks targeted install at confirmation" spec dispatches a popstate to set the target node, then clicks Reinstall with a script click. The click can run before the targeted page has rendered or while the button is still disabled, so no install request is sent and the poll on the install count times out. It failed 1 run in 10 with two workers and on every retry in CI. The spec now waits for the button to be enabled first. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * chore(addons): refresh the voice package digest in the first-party index The Voice add-on became one page with tabs and got a new icon, so the digest in gallery/addons/index.yaml no longer matched the build. Render the entry again with addonpkg catalog over a locally built linux/amd64 package; the main page is now voice. The other entries are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto * test(ui): wait for the target node's Reinstall button after the popstate The "generic deletion blocks targeted install at confirmation" spec dispatches a popstate to set the target node, then waits for a button named Reinstall and clicks it. Until React renders the targeted page, the page still draws the generic Reinstall button. It is enabled and has the same name, so the wait passed on it, the click started a global install, and no node install request was sent. The poll on the install count timed out with the confirm dialog still open. With the CPU throttled to 8x, the spec failed on 20 of 20 runs, and the button it clicked had the title "Reinstall". The spec now waits for the button titled "Reinstall on ". With the throttle it passes 30 of 30 runs. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(ui): give the Studio hand-off spec a budget for the cold bundle load The "a result made through a hand-off records which result it came from" spec opens the Studio add-on straight from an address, twice. Each is a cold load: the page shows "Loading" until the add-on module has run, and the next assertion had the default five seconds. The load takes about two seconds on an idle run and seven to ten under CPU load, so the spec failed on a loaded machine with the hand-off note not yet drawn. With the CPU throttled to 6x and eight busy loops running, it failed on 21 of 30 runs. The first assertion after each load now waits up to 20 seconds, and the spec then passed 30 of 30 under the same load. The assertions are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * fix(addons): carry the identity page styles in the Face add-on The Face page is drawn with the shared identity views, but its style sheet held only the clip and error rules. The page, card, model list, known-faces table, enrol sheet and distance scale classes had no rules. They looked right only when the host had already loaded the style sheet of its own Faces page, which a visit through the add-on drawer does not do. Opened that way, the cards had no gap, the model list showed bullets, and the install buttons ran into the names. The Face sheet now holds the same identity rules as the core sheet, so the page looks the same whichever way it is opened. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * fix(addons): scope the narrow-screen identity rules of the Voice add-on The rules for narrow screens (single-column pairs, smaller tool padding, compact rank rows) were copied into the Voice style sheet without their media query, so they applied at every width. They now sit in a max-width 720px query, and the reduced-motion rule in its own query, as in the core sheet. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * fix(addons): drop the repeated title from the Voice and Face pages The add-on shell already titles the page, so the recognition pages showed the name twice: "Voice" above "Voices", "Face" above "Faces". Keep the supporting line and the model picker, as the Studio page does, and remove the unused title strings. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(ui): replace the Pinned sidebar section with an Apps entry The sidebar no longer lists pinned add-ons. It shows one Apps entry that opens the add-ons drawer, with a Show all link below it that opens the same drawer. Pins are used from the drawer, where a pinned tile keeps its pin and move buttons. The server pins API is unchanged. Remove the PinnedNav component and its locale title, rename the items.addons label to Apps, and update the e2e specs and the add-ons documentation to match. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * test(addons): assert the single shell title on the Voice and Face pages The add-on shell now shows the one title, "Voice" or "Face", and the page no longer repeats it. The tests still looked for the h1 "Voices" and "Faces". Assert that the add-on page has exactly one h1 and that it is the shell title. The core pages keep their own "Voices" and "Faces" titles. Assisted-by: Claude Code:claude-sonnet-5-5 playwright Signed-off-by: Ettore Di Giacinto * fix(addons): drop the repeated title from the Voice Library and Text to speech pages The add-on shell already titles the page "Voice", so these two tabs showed a second heading below it ("Speech voices", "Speak text"). Keep the supporting line, as the Studio and recognition pages do, and remove the unused title strings. The e2e spec now checks that each tab has exactly one h1, the shell title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(ui): list pinned apps under Apps in the sidebar A pinned app now shows as an indented child of the Apps entry, with its icon and title, in pin order. Show all stays the last child. The pins come from GET /api/addons/pins through the navigation context, and the apps from the add-on registry, which the drawer feeds on every list read, so the links follow the drawer. A pin without an installed app is not drawn, nothing extra shows when no app is pinned, and the children are hidden when the sidebar is collapsed to the rail. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * docs(addons): say that pinned apps are listed under Apps in the sidebar The sidebar lists the pinned apps between Apps and Show all, in pin order. The pins section and the navigation section said that pins do not appear in the sidebar. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * fix(ui): remove the Show all link under Apps in the sidebar The link repeated the Apps entry, which opens the same drawer, and looked redundant when no app was pinned. Apps is now the only add-on entry; pinned apps still list under it in pin order. The link, its locale key and the style comment are gone, and the e2e spec and the add-ons docs now say that Apps is the only entry. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(ui): show a first-run card on Apps when nothing is installed An administrator who opens Apps with no add-on installed now sees a short card with three steps: choose a library, browse and install, start the add-on. Its button opens the Browse tab. The card is hidden once an add-on is installed and for anyone who cannot install. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * fix(ui): hide the pinned apps in the icon rail When the window is narrower than 1024 px the sidebar shrinks to the icon rail by CSS alone, without the collapsed class. The pinned app links were still drawn there, with the indent meant for the full sidebar, so their icons sat to the right of the rail and were cut off. Hide the pinned app links in both rail states. The Apps entry keeps its centred icon. The e2e spec checks that the rail shows no pinned app and that every icon is centred at 1280 px collapsed and at 1000 px. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * test(ui): stop the backend redirect test racing the installed list The test reads the redirected URL right after the path changes. The Backends page then loads the installed list and drops a selected backend that is not installed, so the backend parameter was sometimes already gone when the test read it. The mock server has no llama-cpp, and about 15 of 40 repeated runs failed on the backend assertion. The redirect itself was correct, including show_all. Serve the selected backend in the installed list, so the page keeps the selection and the assertions stay as strict as before. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * fix(addons): accept a same-host Referer or Origin for human confirmation Browsers send no Sec-Fetch-* headers over plain HTTP on an address that is not localhost, so creating a resource from a LAN address failed with "Creating a resource requires your own browser session". When Sec-Fetch-Site is present it must still be same-origin with an empty destination, as before. When it is absent, Origin or, failing that, Referer must name the request host and port. A destination or mode that is present must still be the expected one. This matches the rule in BrowserProtection. The credential, token cookie, API key and tool-context checks are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(ui): keep hub entries off the Apps route in the sidebar The Build hub still owned /app/addons through its legacy Add-ons tab, so the sidebar lit both Apps and Build on the Apps page. A tab whose gate is closed now owns no route when the sidebar asks, so only the Apps entry is active there. The hubs keep their own routes, and the legacy navigation still gives /app/addons to the Build hub. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(ui): label the pin control on app tiles and in the detail view The pin was a small icon-only button and hard to find. Each app tile now shows a button reading Pin, or Pinned with a filled pin, and the detail view has the same control in its header. Both toggle the same stored pin. The Apps grid lists pinned apps first, in pin order, then the others. The sidebar list and the server pins API are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * test(addons): check the Voice From a recording tab has one title The Library and Text to speech tabs already assert that the add-on shell title is the only h1. Do the same for the From a recording tab, so a heading added there later fails the navigation spec. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * fix(ui): stop the add-on shell drawing a title over add-on pages The app decides what its page shows. The shell drew the add-on name as a header above every add-on page, so an add-on could not choose its own title. The shell now draws no header on a live, loading or frozen add-on page; each add-on titles its own pages. The unavailable and missing-page states keep the name, because no add-on page is drawn there. The runtime and shell specs now check that the shell adds no page title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * fix(addons): let each add-on draw its own page title The shell no longer titles add-on pages, so each add-on draws one h1 per page. Voice titles its tabs "Voices" (Recognition and From a recording), "Speech voices" (Library) and "Speak text" (Text to speech); "Add a speech voice" already had its own. Face draws "Faces", Studio "Studio" and Talk "Talk". Tables promotes the database list and database name from h2 to h1. Fine-tuning and Quantization already drew a title. The add-on unit tests and the navigation and Studio specs now assert exactly one h1 per page, with the page's own title. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(addons): add a kind to library entries and read unknown kinds tolerantly A library entry now has an optional kind: addon (the default when it is absent), backend or model. Every index written before kinds existed stays valid and its entries are add-ons. A reader keeps the entries of the kinds it knows and skips the others without reading their fields, so a newer kind never makes an older host refuse the whole index. The refresh result and the library state report how many entries were skipped and of which kinds, and the refresh logs a warning. A kind that is not a plain kind name still makes the index malformed. Search lists add-ons unless the caller asks for a kind, and Resolve returns add-ons only, so an add-on install cannot be pointed at a backend image. A library policy may name the signer with an anchored identityRegex instead of an exact identity, so builds from master and from tags both verify. An add-on install refuses a policy that carries one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(backends): install a backend from a library entry A library can now list backends. An entry of kind backend names the image of the backend gallery by digest, and POST /backends/library/install installs it on the local node. The gallery stays the source of truth and its install path is the one used: the library resolves the entry, takes the signature policy from the library, pins the image by digest and calls the same install as the gallery. Installing a name that came from a gallery, another library or by hand is refused and changes nothing. The signature policy of a backend library names the signer with an anchored identityRegex, so images signed by the merge workflow on master and on tags both verify. A pattern that is not anchored is refused. An install is bounded by a size limit per kind: add-ons keep 512 MiB and a backend image defaults to 8 GiB, set with --addons-backend-max-bytes. The limit reads the manifest after the signature check and before any layer is fetched. The metadata of an installed backend records the library it came from. scripts/backends/cmd/backendcatalog renders entries from backend/index.yaml and pins them against the registry. The pilot is the CPU build of local-store, in gallery/backends/index.yaml. The library is registered on the first start only with --addons-backend-library, which is off by default. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * feat(ui): show the kind of library entries and install backends from them The Browse tab lists the entries of every kind this host reads and marks each one as an add-on or a backend. A backend installs through the backend route and not through the add-on lifecycle: the confirmation says that the backend gallery is not changed and that a backend runs native code, and a refusal of the host, such as a name installed from another origin, is shown with its own reason. The details give the kind, the alias and the signer pattern. The library editor can name the signer with a pattern. Without this a save would drop the pattern of a backend library. The Libraries tab reports the entries a refresh skipped because their kind is not read. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * docs(addons): describe backends in the library Document the entry kinds, how a reader skips a kind it does not know, the backend entry format, the signer identity pattern, the install route, the size limit of each kind, the pilot library and the command that renders it. The backend page links to it. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * feat(mcp): install a backend from a library entry The backend library route installs a backend from an add-on library entry, and the admin REST surface and the MCP catalog must stay in step. Add the install_library_backend tool with an in-process and an HTTP client, the route map entry, the safety-rule listing and a skill recipe. Listing library entries stays on the add-on routes, which the assistant does not expose, so no list tool is added. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * fix(ui): carry the add-on library keys in every locale The add-on library and backend library strings existed only in English, so the other locales had no value for them. Add the missing keys of the addons namespace to de, es, id, it, ko, pt-BR and zh-CN. The values are the English text until a translation is contributed. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * Revert "feat(mcp): install a backend from a library entry" This reverts commit 7493cff7270cdec215297680bac57175e1933054. Signed-off-by: Ettore Di Giacinto * Revert "docs(addons): describe backends in the library" This reverts commit 57816ed7e683d68cd674c843a47897f6e6204898. Signed-off-by: Ettore Di Giacinto * Revert "feat(ui): show the kind of library entries and install backends from them" This reverts commit 49b2b151b773122a72d44da965b31d136cd394fc. Signed-off-by: Ettore Di Giacinto * Revert "feat(backends): install a backend from a library entry" This reverts commit 1d8d3e3421fbebff05d3cf208c8e4fffd917993c. Signed-off-by: Ettore Di Giacinto * Revert "feat(addons): add a kind to library entries and read unknown kinds tolerantly" This reverts commit 484813f856edbef7de70619a1a9aeba66e4626be. Signed-off-by: Ettore Di Giacinto * fix(ui): drop the backend library keys from the locales Revert the part of 39f77ae29 that added strings for backend libraries: the kind labels, the backend install texts, the signer identity pattern and the skipped entries notice. The add-on library keys stay. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * feat(ui): add an add-ons control to the chat header Add-on tools were on or off only through the /addons command, with a toast and no state a person could see. The chat header now has an Add-ons button in the style of the MCP dropdown. It lists each running add-on that ships tools with a switch, and a badge shows how many are on. The chat keeps the ids that are on (addonToolIds) next to the older boolean, and both are saved with the chat. A chat that has only the boolean means every add-on. The request sends addon_tools alone when every add-on is on, and adds addon_tool_ids when only some are. /addons turns all on or all off and uses the same state. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * feat(addons): let a chat request name the add-ons whose tools it uses addon_tools=true still offers the tools of every add-on the caller may use. A new addon_tool_ids metadata key, a comma-separated list of add-on ids, narrows that to some add-ons. The list counts only with the opt-in, and a list that names nothing offers no tool rather than all of them. The executor drops the tools of add-ons that are not listed and refuses a call to one as an unknown tool, so a model cannot reach a tool the person turned off. Both keys are consumed so they do not reach the backend. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * test(addons): cover the per-add-on tool list and the chat header control Go tests cover the list form (the list filters the tools offered and the calls allowed) and the unchanged boolean form. Unit tests cover the chat add-on rules. Playwright specs cover the header control: listing only add-ons that ship tools, toggling each one, the count, /addons in step with the switches, and the request metadata. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * test(ui): check each add-on page has one title when not enabled The shell names an add-on only where the add-on page cannot mount (not enabled, failed, no UI, bundle not loaded, no such page). Once the page mounts it draws its own title, including when it says it is not enabled on this server, and the shell draws none. The shell code already works this way; this change states the rule beside the unavailable state and pins it in the specs. The shell spec now expects exactly one title in the unavailable states. The navigation spec checks the Face page without a face model (the add-on title only) and the Face page of a stopped add-on (the shell title only). Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(addons): upgrade add-ons on their own when the setting is on A new auto_upgrade_addons setting (flag LOCALAI_AUTO_UPGRADE_ADDONS, default off) lets the owner upgrade an installed add-on when its library lists a newer version. It follows the backend setting: a startup flag, a runtime setting, and the same schedule as the backend update check (a first pass shortly after start, then every six hours). Each pass fetches the library of every installed add-on, and for each available update builds the same pinned upgrade command an administrator sends and runs it through the same lifecycle path. The channel rules, the signer identity, the digest checks, the SDK compatibility check and the backup before the upgrade all apply unchanged. The only difference is the gate: the owner acts on its own setting, so no administrator is asked. A refused or failed upgrade changes nothing and is logged; a candidate that failed is not retried until the library lists another digest. With the setting off, nothing runs and upgrades stay manual. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * feat(ui): add the Upgrade add-ons automatically toggle to Settings The toggle sits beside Upgrade backends automatically and is off by default. It saves the auto_upgrade_addons runtime setting. The English locale gets the label key, and the Settings test fixture lists the new field. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * test(addons): cover the automatic upgrade setting and document it The setting defaults to off in the config, the runtime settings and the flag, and is read back from the saved settings file. With it on, a real owner upgrades an installed add-on through the manager; with it off, nothing changes, and a change of the setting at run time takes effect on the next pass. The manager specs show the upgrade goes through the same pinned path as the manual one, and that a corrupted package, a failed signature policy, an incompatible release, a missing update checker and a caller who is not an administrator each leave the installed version in place. The new specs are added to the lifecycle case inventory. The add-ons page describes the setting, its schedule and its default. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * refactor(addons): name the library entry identifier id The library entry of an add-on used addonId while the shared envelope and the model entries use id. Release now has the field ID, written as id, and every writer emits it: the index encoder, the package workflow templates, the catalog generator and gallery/addons/index.yaml. The digests in the index did not change. DecodeIndex still reads addonId as a legacy alias for one release, so an existing library and a library served by an older tool keep working. An entry that holds both names is refused. The Apps UI reads id and falls back to addonId. Assisted-by: Claude Code:claude-sonnet-5-5 go node playwright Signed-off-by: Ettore Di Giacinto * test(addons): use the id field in the auto-upgrade test release The auto-upgrade test arrived after the identifier rename and still named the release field AddonID, so the application tests did not build. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(addons): annotate the auto-upgrade actor label for gosec The actor label looks like a credential to gosec G101. It is a fixed name for upgrades the owner starts itself. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * fix(schema): persist the per-chat add-on tool fields The chat header persists addonTools and addonToolIds, but Conversation had no fields for them, so the server dropped them on save. Add both and update the generated swagger files by hand because swag is not installed here. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto * test(ui): count the add-ons auto-upgrade setting in the settings groups The setting sits in the Backends and galleries group, once. The page now lists 54 settings plus the three branding images, so the spec expects 57. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * test(ui): pause the fake clock ahead of now in the clear-all spec The fake clock runs from install, so pausing at the current time failed with 'Cannot fast-forward to the past' on a slow runner. The save itself is synchronous and was not lost. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * test(ui): hold the Studio capabilities reply longer in the skeleton spec 800 ms was shorter than a slow runner needs to render the page, so the skeleton was gone before the check. The product shows it. Assisted-by: Claude Code:claude-sonnet-5-5 node playwright Signed-off-by: Ettore Di Giacinto * feat(i18n): add the French add-ons locale Master added the fr locale after this branch forked. The add-ons unit test requires every locale to carry the add-ons namespace, so fr needs its own addons.json. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto --------- Signed-off-by: Ettore Di Giacinto Co-authored-by: Ettore Di Giacinto --- .agents/backend-signing.md | 7 + .agents/localai-assistant-mcp.md | 26 + .github/workflows/addons-package.yml | 426 ++ .gitignore | 1 + addons/face/cmd/face/main.go | 15 + addons/face/manifest.json | 25 + addons/face/package/platforms.yaml | 38 + addons/face/ui/.gitignore | 2 + addons/face/ui/build.mjs | 13 + addons/face/ui/icon.png | Bin 0 -> 6999 bytes addons/face/ui/package.json | 8 + addons/face/ui/src/FaceRecognition.jsx | 116 + addons/face/ui/src/MoreFaceTools.jsx | 246 + addons/face/ui/src/api.js | 35 + addons/face/ui/src/ctx.jsx | 67 + addons/face/ui/src/i18n/en.js | 198 + addons/face/ui/src/index.jsx | 27 + addons/face/ui/src/lib/model.js | 8 + addons/face/ui/src/style.css | 242 + addons/face/ui/test/model.test.mjs | 17 + addons/face/ui/test/recognition.test.mjs | 86 + addons/face/ui/test/render.test.mjs | 75 + addons/face/ui/test/run.mjs | 23 + addons/fine-tuning/cmd/fine-tuning/main.go | 21 + addons/fine-tuning/internal/tools/journal.go | 142 + .../fine-tuning/internal/tools/suite_test.go | 14 + addons/fine-tuning/internal/tools/tools.go | 557 ++ .../fine-tuning/internal/tools/tools_test.go | 527 ++ addons/fine-tuning/manifest.json | 285 + addons/fine-tuning/package/platforms.yaml | 38 + addons/fine-tuning/ui/.gitignore | 2 + addons/fine-tuning/ui/build.mjs | 13 + addons/fine-tuning/ui/icon.png | Bin 0 -> 5362 bytes addons/fine-tuning/ui/package.json | 8 + addons/fine-tuning/ui/src/FineTune.jsx | 1350 ++++ addons/fine-tuning/ui/src/LossChart.jsx | 169 + addons/fine-tuning/ui/src/api.js | 37 + addons/fine-tuning/ui/src/ctx.jsx | 68 + addons/fine-tuning/ui/src/i18n/en.js | 300 + addons/fine-tuning/ui/src/index.jsx | 46 + addons/fine-tuning/ui/src/lib/model.js | 8 + addons/fine-tuning/ui/src/lib/progress.js | 22 + addons/fine-tuning/ui/src/style.css | 271 + addons/fine-tuning/ui/test/load.mjs | 49 + addons/fine-tuning/ui/test/model.test.mjs | 17 + addons/fine-tuning/ui/test/page.test.mjs | 248 + addons/fine-tuning/ui/test/progress.test.mjs | 32 + addons/fine-tuning/ui/test/render.test.mjs | 51 + addons/fine-tuning/ui/test/run.mjs | 23 + addons/internal/host-compat/package.json | 8 + .../src/components/ConfirmDialog.jsx | 4 + .../host-compat/src/components/Icon.jsx | 8 + .../host-compat/src/components/Lightbox.jsx | 4 + .../src/components/LoadingSpinner.jsx | 4 + .../host-compat/src/components/Modal.jsx | 4 + .../src/components/ModelSelector.jsx | 4 + .../host-compat/src/components/PageHeader.jsx | 4 + .../src/components/audio/WaveformPlayer.jsx | 4 + .../src/components/biometrics/MediaInput.jsx | 4 + .../src/components/home/HomeModelPicker.jsx | 4 + .../host-compat/src/components/home/memory.js | 54 + .../host-compat/src/context/AuthContext.jsx | 8 + .../host-compat/src/hooks/useAudioPeaks.js | 71 + .../host-compat/src/hooks/useMCPClient.js | 6 + .../host-compat/src/hooks/useMediaCapture.js | 10 + .../host-compat/src/hooks/useModels.js | 6 + .../host-compat/src/hooks/useObjectUrl.js | 6 + .../host-compat/src/hooks/useOperations.js | 6 + .../host-compat/src/hooks/useResources.js | 6 + .../host-compat/src/hooks/useVoiceProfiles.js | 6 + addons/internal/host-compat/src/i18n.js | 92 + .../host-compat/src/modules/jsx-runtime.js | 10 + .../host-compat/src/modules/react-dom.js | 7 + .../host-compat/src/modules/react-i18next.js | 18 + .../src/modules/react-router-dom.js | 90 + .../internal/host-compat/src/modules/react.js | 10 + addons/internal/host-compat/src/route.js | 58 + addons/internal/host-compat/src/runtime.js | 23 + addons/internal/host-compat/src/ui.js | 14 + addons/internal/host-compat/src/utils/api.js | 221 + .../host-compat/src/utils/basePath.js | 20 + .../host-compat/src/utils/capabilities.js | 37 + .../host-compat/src/utils/clipboard.js | 81 + .../internal/host-compat/src/utils/config.js | 172 + .../host-compat/src/utils/editorNav.js | 15 + .../internal/host-compat/src/utils/format.js | 74 + .../host-compat/src/utils/lastModel.js | 15 + .../host-compat/src/utils/mcpClientStorage.js | 92 + .../host-compat/src/utils/modelBudget.js | 41 + .../host-compat/src/utils/modelLedger.js | 113 + addons/internal/host-compat/test/api.test.mjs | 127 + .../internal/host-compat/test/copies.test.mjs | 48 + addons/internal/host-compat/test/host.mjs | 70 + .../internal/host-compat/test/i18n.test.mjs | 66 + addons/internal/host-compat/test/loader.mjs | 28 + addons/internal/host-compat/test/parity.mjs | 85 + addons/internal/host-compat/test/register.mjs | 5 + .../internal/host-compat/test/route.test.mjs | 40 + .../internal/host-compat/test/router.test.mjs | 98 + addons/internal/host-compat/test/run.mjs | 23 + addons/internal/hostclient/client.go | 106 + addons/internal/hostclient/client_test.go | 74 + addons/internal/hostclient/suite_test.go | 14 + .../identity-ui/src/BoundingBoxCanvas.jsx | 57 + addons/internal/identity-ui/src/ClipInput.jsx | 225 + .../identity-ui/src/DistanceScale.jsx | 45 + .../internal/identity-ui/src/EnrolSheet.jsx | 148 + addons/internal/identity-ui/src/ErrorNote.jsx | 27 + addons/internal/identity-ui/src/KnownList.jsx | 109 + .../internal/identity-ui/src/ModelNeeded.jsx | 99 + addons/internal/identity-ui/src/Verdict.jsx | 23 + addons/internal/identity-ui/src/Workbench.jsx | 345 + addons/internal/identity-ui/src/identity.js | 146 + .../identity-ui/src/referenceAudio.js | 42 + addons/internal/identity-ui/src/registry.js | 122 + .../identity-ui/test/identity.test.mjs | 49 + addons/internal/idle/idle.go | 49 + addons/internal/idle/idle_test.go | 45 + addons/internal/idle/suite_test.go | 14 + addons/internal/protocol/protocol.go | 211 + addons/internal/protocol/run_test.go | 19 + addons/internal/protocol/suite_test.go | 14 + addons/internal/tools-ui/src/Checks.jsx | 77 + addons/internal/tools-ui/src/JobLog.jsx | 76 + addons/internal/tools-ui/src/StageLine.jsx | 24 + addons/internal/tools-ui/src/ToolOff.jsx | 27 + addons/internal/tools-ui/src/ToolSteps.jsx | 25 + addons/internal/tools-ui/src/format.js | 8 + addons/internal/tools-ui/src/sse.js | 42 + addons/internal/tools-ui/src/tools.css | 268 + addons/internal/tools-ui/src/tools.js | 443 ++ addons/internal/tools-ui/test/copies.test.mjs | 26 + .../internal/tools-ui/test/fixture/.gitignore | 2 + .../tools-ui/test/fixture/src/index.jsx | 40 + addons/internal/tools-ui/test/render.test.mjs | 109 + addons/internal/tools-ui/test/sse.test.mjs | 28 + addons/internal/tools-ui/test/tools.test.mjs | 204 + addons/internal/uibuild/package.json | 8 + .../compat/components/Badge.jsx | 1 + .../compat/modules/jsx-runtime.js | 4 + .../compat/modules/react-dom.js | 1 + .../compat/modules/react-i18next.js | 1 + .../compat/modules/react-router-dom.js | 1 + .../fixture-compat/compat/modules/react.js | 1 + .../host-compat-fixture-runtime.js | 1 + .../uibuild/test/fixture-compat/src/Page.jsx | 10 + .../uibuild/test/fixture-compat/src/index.jsx | 11 + .../uibuild/test/fixture-compat/src/page.css | 1 + .../uibuild/test/fixture-compat/src/style.css | 1 + .../uibuild/test/fixture-core/src/index.jsx | 2 + .../uibuild/test/fixture-core/src/style.css | 1 + .../uibuild/test/fixture-icon/icon.png | Bin 0 -> 4516 bytes .../uibuild/test/fixture-icon/src/index.jsx | 4 + .../uibuild/test/fixture-icon/src/style.css | 1 + .../uibuild/test/fixture-react/src/index.jsx | 2 + .../uibuild/test/fixture-react/src/style.css | 1 + .../uibuild/test/fixture/src/index.jsx | 4 + .../uibuild/test/fixture/src/style.css | 1 + addons/internal/uibuild/test/uibuild.test.mjs | 86 + addons/internal/uibuild/uibuild.mjs | 155 + addons/quantization/cmd/quantization/main.go | 21 + addons/quantization/internal/tools/journal.go | 138 + .../quantization/internal/tools/suite_test.go | 14 + addons/quantization/internal/tools/tools.go | 421 ++ .../quantization/internal/tools/tools_test.go | 447 ++ addons/quantization/manifest.json | 163 + addons/quantization/package/platforms.yaml | 38 + addons/quantization/ui/.gitignore | 2 + addons/quantization/ui/build.mjs | 13 + addons/quantization/ui/icon.png | Bin 0 -> 8510 bytes addons/quantization/ui/package.json | 8 + addons/quantization/ui/src/Quantize.jsx | 598 ++ addons/quantization/ui/src/api.js | 30 + addons/quantization/ui/src/ctx.jsx | 68 + addons/quantization/ui/src/i18n/en.js | 181 + addons/quantization/ui/src/index.jsx | 46 + addons/quantization/ui/src/lib/model.js | 8 + addons/quantization/ui/src/lib/progress.js | 15 + addons/quantization/ui/src/style.css | 271 + addons/quantization/ui/test/load.mjs | 49 + addons/quantization/ui/test/model.test.mjs | 17 + addons/quantization/ui/test/page.test.mjs | 220 + addons/quantization/ui/test/progress.test.mjs | 28 + addons/quantization/ui/test/render.test.mjs | 51 + addons/quantization/ui/test/run.mjs | 23 + addons/studio/cmd/studio/main.go | 15 + addons/studio/manifest.json | 25 + addons/studio/package/platforms.yaml | 38 + addons/studio/ui/.gitignore | 2 + addons/studio/ui/build.mjs | 15 + addons/studio/ui/icon.png | Bin 0 -> 8844 bytes addons/studio/ui/package.json | 8 + addons/studio/ui/src/addon.jsx | 27 + .../ui}/src/components/AnimationOptions.jsx | 0 .../ui}/src/components/AnimationViewer.jsx | 0 .../studio/ui}/src/components/GlbViewer.jsx | 0 .../ui}/src/components/RequestPanel.jsx | 0 .../ui}/src/components/audio/Spectrogram.jsx | 0 .../ui}/src/components/studio/HandoffNote.jsx | 0 .../ui}/src/components/studio/InstallNote.jsx | 0 .../ui}/src/components/studio/LineageView.jsx | 0 .../ui}/src/components/studio/StarButton.jsx | 0 .../src/components/studio/StudioComposer.jsx | 0 .../ui}/src/components/studio/Timeline.jsx | 0 .../src/components/studio/UpscaleComposer.jsx | 0 .../ui}/src/components/studio/WorkMasonry.jsx | 0 .../ui}/src/components/studio/WorkThumb.jsx | 0 .../ui}/src/components/studio/Workspace.jsx | 0 .../ui}/src/components/studio/studio.css | 0 .../ui}/src/components/studio/workspace.css | 4 + .../studio/ui}/src/hooks/use3DHistory.js | 0 .../studio/ui}/src/hooks/useMediaHistory.js | 0 .../ui}/src/hooks/useModelSuggestion.js | 0 .../ui}/src/hooks/useSoundExpansionModel.js | 0 .../studio/ui}/src/hooks/useSpectrogram.js | 3 +- .../studio/ui}/src/hooks/useStudioHandoff.js | 4 +- .../studio/ui}/src/hooks/useStudioWork.js | 0 .../studio/ui}/src/hooks/useWorkspace.js | 0 addons/studio/ui/src/i18n/media.de.json | 195 + addons/studio/ui/src/i18n/media.en.json | 863 +++ addons/studio/ui/src/i18n/media.es.json | 195 + addons/studio/ui/src/i18n/media.id.json | 242 + addons/studio/ui/src/i18n/media.it.json | 195 + addons/studio/ui/src/i18n/media.ko.json | 195 + addons/studio/ui/src/i18n/media.pt-BR.json | 464 ++ addons/studio/ui/src/i18n/media.zh-CN.json | 195 + addons/studio/ui/src/index.jsx | 20 + .../studio/ui}/src/pages/AudioTransform.jsx | 5 +- .../studio/ui}/src/pages/Diarization.jsx | 0 .../studio/ui}/src/pages/ImageGen.jsx | 0 .../studio/ui}/src/pages/Sound.jsx | 0 .../studio/ui}/src/pages/Studio.jsx | 0 .../studio/ui}/src/pages/StudioOverview.jsx | 0 .../studio/ui}/src/pages/TTS.jsx | 0 .../studio/ui}/src/pages/ThreeDGen.jsx | 5 +- .../studio/ui}/src/pages/VideoGen.jsx | 0 addons/studio/ui/src/style.css | 1 + .../studio/ui}/src/utils/animationGlb.js | 0 .../studio/ui}/src/utils/diarization.js | 0 .../studio/ui}/src/utils/fft.js | 0 .../studio/ui}/src/utils/glb.js | 0 .../studio/ui}/src/utils/mediaHistoryStore.js | 0 .../studio/ui}/src/utils/studioWork.js | 0 .../studio/ui}/src/utils/upscale.js | 0 .../studio/ui}/src/utils/voiceEnrollments.js | 0 .../studio/ui/test/diarization.test.mjs | 2 +- addons/studio/ui/test/parity.test.mjs | 53 + addons/studio/ui/test/render.test.mjs | 61 + addons/studio/ui/test/run.mjs | 23 + .../studio/ui/test/studioWork.test.mjs | 4 +- .../studio/ui/test/upscale.test.mjs | 7 +- addons/tables/cmd/tables-faults/main.go | 64 + addons/tables/cmd/tables/main.go | 17 + addons/tables/internal/server/export.go | 229 + addons/tables/internal/server/export_test.go | 265 + addons/tables/internal/server/rows.go | 179 + addons/tables/internal/server/run.go | 70 + .../internal/server/schema_authority_test.go | 57 + .../internal/server/server_suite_test.go | 14 + addons/tables/internal/server/tables.go | 422 ++ .../internal/store/capacity_internal_test.go | 66 + addons/tables/internal/store/databases.go | 734 ++ .../tables/internal/store/databases_test.go | 307 + addons/tables/internal/store/export.go | 359 + addons/tables/internal/store/export_test.go | 288 + addons/tables/internal/store/migrate.go | 286 + .../internal/store/migrate_internal_test.go | 26 + addons/tables/internal/store/migrate_test.go | 156 + addons/tables/internal/store/query.go | 847 +++ addons/tables/internal/store/query_test.go | 709 ++ addons/tables/internal/store/receipts.go | 100 + addons/tables/internal/store/receipts_test.go | 142 + addons/tables/internal/store/revisions.go | 67 + addons/tables/internal/store/rows.go | 636 ++ addons/tables/internal/store/rows_test.go | 459 ++ addons/tables/internal/store/schema.go | 951 +++ addons/tables/internal/store/schema_test.go | 538 ++ .../tables/internal/store/store_suite_test.go | 14 + addons/tables/manifest.json | 638 ++ addons/tables/package/platforms.yaml | 34 + addons/tables/ui/.gitignore | 2 + addons/tables/ui/build.mjs | 92 + addons/tables/ui/icon.png | Bin 0 -> 7078 bytes addons/tables/ui/package.json | 11 + addons/tables/ui/src/Database.jsx | 153 + addons/tables/ui/src/Databases.jsx | 193 + addons/tables/ui/src/Export.jsx | 77 + addons/tables/ui/src/Filters.jsx | 71 + addons/tables/ui/src/Rows.jsx | 229 + addons/tables/ui/src/Schema.jsx | 221 + addons/tables/ui/src/SelectDatabase.jsx | 115 + addons/tables/ui/src/Sharing.jsx | 110 + addons/tables/ui/src/TableView.jsx | 231 + addons/tables/ui/src/api.js | 35 + addons/tables/ui/src/ctx.js | 34 + addons/tables/ui/src/fields.jsx | 61 + addons/tables/ui/src/hooks.js | 56 + addons/tables/ui/src/i18n/en.js | 261 + addons/tables/ui/src/i18n/index.js | 6 + addons/tables/ui/src/i18n/locales.js | 1726 +++++ addons/tables/ui/src/index.jsx | 71 + addons/tables/ui/src/lib/batch.js | 66 + addons/tables/ui/src/lib/codecs.js | 199 + addons/tables/ui/src/lib/errors.js | 28 + addons/tables/ui/src/lib/export.js | 31 + addons/tables/ui/src/lib/filters.js | 90 + addons/tables/ui/src/lib/query.js | 54 + addons/tables/ui/src/lib/retry.js | 34 + addons/tables/ui/src/style.css | 127 + addons/tables/ui/src/ui.jsx | 89 + addons/tables/ui/test/batch.test.mjs | 65 + addons/tables/ui/test/codecs.test.mjs | 117 + addons/tables/ui/test/filters.test.mjs | 77 + addons/tables/ui/test/i18n.test.mjs | 25 + addons/tables/ui/test/lint.test.mjs | 42 + addons/tables/ui/test/render.test.mjs | 87 + addons/tables/ui/test/retry.test.mjs | 42 + addons/tables/ui/test/run.mjs | 23 + addons/talk/cmd/talk/main.go | 15 + addons/talk/manifest.json | 25 + addons/talk/package/platforms.yaml | 38 + addons/talk/ui/.gitignore | 2 + addons/talk/ui/build.mjs | 15 + addons/talk/ui/icon.png | Bin 0 -> 4687 bytes addons/talk/ui/package.json | 8 + addons/talk/ui/src/addon.jsx | 19 + .../ui}/src/components/ClientMCPDropdown.jsx | 0 .../ui}/src/components/VoiceVisualizer.jsx | 0 .../ui}/src/components/talk/SessionSheet.jsx | 0 .../talk/ui/src/i18n/talk.en.json | 0 addons/talk/ui/src/index.jsx | 17 + .../talk/ui}/src/pages/Talk.jsx | 2 +- .../talk/ui}/src/pages/talk.css | 2 +- addons/talk/ui/src/style.css | 1 + addons/talk/ui/test/parity.test.mjs | 39 + addons/talk/ui/test/render.test.mjs | 62 + addons/talk/ui/test/run.mjs | 23 + addons/voice/cmd/voice/main.go | 15 + addons/voice/manifest.json | 25 + addons/voice/package/platforms.yaml | 38 + addons/voice/ui/.gitignore | 2 + addons/voice/ui/build.mjs | 13 + addons/voice/ui/icon.png | Bin 0 -> 4516 bytes addons/voice/ui/package.json | 8 + addons/voice/ui/src/MoreVoiceTools.jsx | 212 + addons/voice/ui/src/Speech.jsx | 96 + addons/voice/ui/src/VoiceHub.jsx | 72 + addons/voice/ui/src/VoiceLibrary.jsx | 459 ++ addons/voice/ui/src/VoiceProfileCreate.jsx | 252 + addons/voice/ui/src/VoiceRecognition.jsx | 135 + addons/voice/ui/src/WaveformStrip.jsx | 107 + addons/voice/ui/src/api.js | 41 + addons/voice/ui/src/ctx.jsx | 67 + addons/voice/ui/src/i18n/en.js | 348 + addons/voice/ui/src/index.jsx | 61 + addons/voice/ui/src/lib/clipboard.js | 81 + addons/voice/ui/src/lib/model.js | 8 + addons/voice/ui/src/speechRequest.js | 21 + addons/voice/ui/src/style.css | 272 + addons/voice/ui/test/library.test.mjs | 96 + addons/voice/ui/test/model.test.mjs | 17 + addons/voice/ui/test/recognition.test.mjs | 63 + addons/voice/ui/test/render.test.mjs | 96 + addons/voice/ui/test/run.mjs | 23 + addons/voice/ui/test/speech.test.mjs | 39 + core/application/addon_admin_auth_test.go | 85 + core/application/addon_admin_test.go | 20 + core/application/addon_auto_upgrade.go | 49 + core/application/addon_auto_upgrade_test.go | 122 + core/application/addon_engine_jobs_test.go | 34 + core/application/addon_gateway.go | 425 ++ core/application/addon_host_ops_auth_test.go | 32 + core/application/addon_host_ops_test.go | 29 + .../addon_pin_store_internal_test.go | 34 + core/application/addon_pin_sweep_test.go | 64 + core/application/addon_pins.go | 136 + core/application/addon_pins_test.go | 105 + core/application/addons_auth_test.go | 338 + core/application/addons_other_test.go | 24 + core/application/addons_test.go | 268 + core/application/application.go | 178 +- .../runtime_settings_branding_test.go | 6 + core/application/startup.go | 58 +- core/application/startup_auth_test.go | 58 + core/application/upgrade_checker.go | 11 +- core/cli/agent_worker.go | 16 +- core/cli/agent_worker_mcpfilter_test.go | 76 + core/cli/auto_upgrade_addons_test.go | 34 + core/cli/run.go | 34 +- core/cli/run_addon_host_ops_test.go | 19 + core/cli/run_addons.go | 67 + core/cli/run_addons_test.go | 131 + core/config/addon_host_ops.go | 44 + core/config/addon_host_ops_test.go | 50 + core/config/application_config.go | 70 + core/config/application_config_test.go | 24 + core/config/runtime_settings.go | 1 + core/config/runtime_settings_registry.go | 4 + core/http/app.go | 20 + core/http/auth/addons.go | 150 + core/http/auth/addons_test.go | 123 + core/http/auth/addons_users.go | 166 + core/http/auth/core_calls_test.go | 86 + core/http/auth/features.go | 41 + core/http/auth/permissions.go | 3 +- core/http/endpoints/addons/apps.go | 172 + core/http/endpoints/addons/apps_test.go | 407 + core/http/endpoints/addons/artifacts.go | 100 + core/http/endpoints/addons/artifacts_test.go | 465 ++ core/http/endpoints/addons/assets.go | 67 + core/http/endpoints/addons/assets_test.go | 224 + .../endpoints/addons/browserguard_test.go | 66 + core/http/endpoints/addons/builtin.go | 45 + core/http/endpoints/addons/builtin_test.go | 52 + core/http/endpoints/addons/catalogui_test.go | 341 + core/http/endpoints/addons/forwarding_test.go | 140 + core/http/endpoints/addons/gateway.go | 316 + .../endpoints/addons/gateway_auth_test.go | 80 + core/http/endpoints/addons/gateway_test.go | 88 + .../endpoints/addons/human_surface_test.go | 48 + .../endpoints/addons/installrefusal_test.go | 164 + core/http/endpoints/addons/pins.go | 124 + core/http/endpoints/addons/pins_test.go | 114 + core/http/endpoints/addons/proposals.go | 358 + core/http/endpoints/addons/proposals_test.go | 867 +++ core/http/endpoints/addons/recovery.go | 324 + core/http/endpoints/addons/resources.go | 436 ++ .../http/endpoints/addons/resulterror_test.go | 272 + core/http/endpoints/addons/routes_test.go | 366 + core/http/endpoints/addons/tools_test.go | 189 + core/http/endpoints/addons/updates.go | 197 + core/http/endpoints/addons/updates_test.go | 416 ++ core/http/endpoints/anthropic/messages.go | 6 + .../endpoints/localai/addon_engine_jobs.go | 332 + .../localai/addon_engine_models_test.go | 123 + .../endpoints/localai/api_instructions.go | 2 + .../localai/api_instructions_test.go | 3 +- core/http/endpoints/localai/finetune.go | 44 +- core/http/endpoints/localai/mcp.go | 12 +- .../localai/mcp_addon_tools_internal_test.go | 38 + core/http/endpoints/localai/quantization.go | 42 +- core/http/endpoints/mcp/addon_tools.go | 312 + core/http/endpoints/mcp/tools.go | 24 +- core/http/endpoints/openai/chat.go | 27 +- .../openai/chat_mcp_auto_gate_test.go | 2 +- core/http/endpoints/openai/evidence_test.go | 2 +- .../http/endpoints/openresponses/responses.go | 12 +- core/http/react-ui/e2e/add-node.spec.js | 10 +- .../react-ui/e2e/addon-navigation.spec.js | 801 ++ core/http/react-ui/e2e/addons-fixtures.js | 179 + .../react-ui/e2e/addons-foundation.spec.js | 177 + core/http/react-ui/e2e/addons-gallery.spec.js | 687 ++ .../react-ui/e2e/addons-proposals.spec.js | 332 + .../http/react-ui/e2e/addons-recovery.spec.js | 262 + core/http/react-ui/e2e/addons-runtime.spec.js | 483 ++ core/http/react-ui/e2e/addons-shell.spec.js | 100 + core/http/react-ui/e2e/addons-slots.spec.js | 409 + core/http/react-ui/e2e/addons-upgrade.spec.js | 335 + .../http/react-ui/e2e/audio-transform.spec.js | 3 + core/http/react-ui/e2e/build-landing.spec.js | 3 +- core/http/react-ui/e2e/chat-addons.spec.js | 160 + core/http/react-ui/e2e/chat-fixtures.js | 3 + .../react-ui/e2e/chat-history-sync.spec.js | 8 +- .../react-ui/e2e/chat-pdf-attachment.spec.js | 3 +- core/http/react-ui/e2e/chrome-audit.spec.js | 3 + .../react-ui/e2e/diarization-profiles.spec.js | 5 +- .../e2e/discover-search-focus.spec.js | 11 +- core/http/react-ui/e2e/global-setup.js | 6 + core/http/react-ui/e2e/home-console.spec.js | 26 +- core/http/react-ui/e2e/hub-tabs.spec.js | 16 +- core/http/react-ui/e2e/identity.spec.js | 6 +- .../react-ui/e2e/legacy-redirects.spec.js | 40 + core/http/react-ui/e2e/media-history.spec.js | 75 +- core/http/react-ui/e2e/navigation.spec.js | 29 +- core/http/react-ui/e2e/no-edge-rails.spec.js | 3 + .../e2e/nodes-fleet-dashboard.spec.js | 8 +- core/http/react-ui/e2e/not-found.spec.js | 4 +- .../react-ui/e2e/page-header-no-crumb.spec.js | 1 - .../react-ui/e2e/page-render-smoke.spec.js | 30 +- core/http/react-ui/e2e/settings-fixtures.js | 2 +- core/http/react-ui/e2e/settings-page.spec.js | 6 +- core/http/react-ui/e2e/studio-addon.js | 73 + core/http/react-ui/e2e/studio-addons.spec.js | 111 + .../http/react-ui/e2e/studio-composer.spec.js | 30 +- core/http/react-ui/e2e/studio-fixtures.js | 6 +- core/http/react-ui/e2e/studio-lineage.spec.js | 10 +- .../http/react-ui/e2e/studio-overview.spec.js | 8 +- .../react-ui/e2e/studio-transform.spec.js | 3 + .../react-ui/e2e/studio-workbench.spec.js | 3 + .../react-ui/e2e/studio-workspace.spec.js | 8 +- .../react-ui/e2e/tables-acceptance.spec.js | 245 + .../react-ui/e2e/tables-artifacts.spec.js | 404 + core/http/react-ui/e2e/tables-ui.spec.js | 937 +++ core/http/react-ui/e2e/talk.spec.js | 4 + .../react-ui/e2e/threed-animation.spec.js | 3 + core/http/react-ui/e2e/threed-gen.spec.js | 5 +- core/http/react-ui/e2e/voice-library.spec.js | 28 +- core/http/react-ui/inline-style-baseline.txt | 2 +- core/http/react-ui/package.json | 1 + .../http/react-ui/playwright.addons.config.js | 27 + core/http/react-ui/playwright.config.js | 12 + core/http/react-ui/playwright.csv.config.js | 31 + core/http/react-ui/playwright.esm.config.js | 30 + .../react-ui/playwright.foundation.config.js | 27 + .../react-ui/playwright.recovery.config.js | 30 + core/http/react-ui/playwright.slots.config.js | 30 + .../playwright.tables-acceptance.config.js | 27 + .../http/react-ui/playwright.tables.config.js | 31 + core/http/react-ui/playwright.tools.config.js | 31 + .../react-ui/playwright.updates.config.js | 30 + .../react-ui/public/locales/de/addons.json | 538 ++ core/http/react-ui/public/locales/de/nav.json | 9 +- .../react-ui/public/locales/en/addons.json | 538 ++ .../react-ui/public/locales/en/admin.json | 1 + .../http/react-ui/public/locales/en/chat.json | 15 +- core/http/react-ui/public/locales/en/nav.json | 9 +- .../react-ui/public/locales/es/addons.json | 538 ++ core/http/react-ui/public/locales/es/nav.json | 9 +- .../react-ui/public/locales/fr/addons.json | 542 ++ .../react-ui/public/locales/id/addons.json | 538 ++ core/http/react-ui/public/locales/id/nav.json | 9 +- .../react-ui/public/locales/it/addons.json | 538 ++ core/http/react-ui/public/locales/it/nav.json | 9 +- .../react-ui/public/locales/ko/addons.json | 538 ++ core/http/react-ui/public/locales/ko/nav.json | 9 +- .../react-ui/public/locales/pt-BR/addons.json | 538 ++ .../react-ui/public/locales/pt-BR/nav.json | 9 +- .../react-ui/public/locales/zh-CN/addons.json | 538 ++ .../react-ui/public/locales/zh-CN/nav.json | 9 +- core/http/react-ui/src/App.css | 10 + .../src/addons/AddonErrorBoundary.jsx | 35 + core/http/react-ui/src/addons/AddonRoute.jsx | 231 + core/http/react-ui/src/addons/AddonSlot.jsx | 199 + .../react-ui/src/addons/ApprovalFrame.jsx | 108 + .../http/react-ui/src/addons/ArtifactCard.jsx | 129 + .../http/react-ui/src/addons/ProposalCard.jsx | 159 + .../http/react-ui/src/addons/SlotBoundary.jsx | 24 + core/http/react-ui/src/addons/approval.js | 84 + .../http/react-ui/src/addons/artifactFacts.js | 68 + core/http/react-ui/src/addons/cssScope.js | 283 + .../react-ui/src/addons/hostComponents.jsx | 63 + core/http/react-ui/src/addons/hostServices.js | 85 + core/http/react-ui/src/addons/loader.js | 265 + .../react-ui/src/addons/proposalResult.js | 62 + core/http/react-ui/src/addons/registry.jsx | 297 + .../react-ui/src/addons/registryContext.js | 7 + .../http/react-ui/src/addons/registryState.js | 98 + .../http/react-ui/src/addons/resourceCalls.js | 135 + core/http/react-ui/src/addons/runtime.js | 154 + core/http/react-ui/src/addons/sdk.js | 479 ++ core/http/react-ui/src/addons/shell.css | 121 + core/http/react-ui/src/addons/slots.js | 223 + core/http/react-ui/src/addons/useAddonPage.js | 16 + .../react-ui/src/addons/useAddonRegistry.js | 102 + .../src/components/AppsPinnedLinks.jsx | 48 + .../react-ui/src/components/ConfirmDialog.jsx | 30 +- .../react-ui/src/components/SideSheet.jsx | 9 +- core/http/react-ui/src/components/Sidebar.jsx | 45 +- .../src/components/addons/AddonBits.jsx | 75 + .../src/components/addons/AddonTile.jsx | 43 + .../src/components/addons/AppsView.jsx | 292 + .../src/components/addons/BrowseView.jsx | 170 + .../src/components/addons/InstalledView.jsx | 423 ++ .../src/components/addons/LibrariesView.jsx | 11 + .../src/components/addons/PinControl.jsx | 25 + .../src/components/addons/RecoveryPanel.jsx | 289 + .../src/components/addons/actionBook.js | 56 + .../src/components/addons/addonState.js | 124 + .../react-ui/src/components/addons/addons.css | 226 + .../react-ui/src/components/addons/errors.js | 35 + .../src/components/addons/lifecycleFlow.js | 56 + .../src/components/addons/recoveryState.js | 102 + .../src/components/addons/updateState.js | 93 + .../src/components/addons/useAddonActions.js | 136 + .../src/components/catalog/CatalogEmpty.jsx | 14 + .../src/components/catalog/CatalogPager.jsx | 23 + .../components/catalog/CatalogSearchBox.jsx | 24 + .../src/components/catalog/CatalogView.jsx | 123 + .../src/components/catalog/SourcesView.jsx | 221 + .../src/components/catalog/catalog.css | 15 + .../src/components/catalog/catalogState.js | 183 + .../src/components/chat/ActivityFold.jsx | 61 +- .../src/components/chat/AddonToolsMenu.jsx | 67 + .../src/components/chat/ChatHeader.jsx | 3 +- .../src/components/chat/ChatMessage.jsx | 5 +- .../src/components/chat/chatActions.js | 4 +- .../src/components/home/HomeComposer.jsx | 4 +- .../src/components/home/homeActions.js | 8 +- .../react-ui/src/components/hub/HubLayout.jsx | 19 +- .../react-ui/src/components/hub/hubConfig.js | 86 +- .../http/react-ui/src/context/AuthContext.jsx | 5 +- .../src/context/NavigationContext.jsx | 59 + core/http/react-ui/src/context/authState.js | 6 + core/http/react-ui/src/hooks/useChat.js | 35 +- core/http/react-ui/src/i18n/index.js | 3 +- core/http/react-ui/src/pages/Addons.jsx | 96 + core/http/react-ui/src/pages/Backends.jsx | 57 +- .../http/react-ui/src/pages/BuildOverview.jsx | 6 +- core/http/react-ui/src/pages/Chat.jsx | 33 +- core/http/react-ui/src/pages/Home.jsx | 13 +- core/http/react-ui/src/pages/NotFound.jsx | 9 +- core/http/react-ui/src/pages/Settings.jsx | 3 + core/http/react-ui/src/pages/chat.css | 4 + core/http/react-ui/src/router.jsx | 157 +- core/http/react-ui/src/utils/addonTiles.js | 109 + core/http/react-ui/src/utils/addonsApi.js | 235 + core/http/react-ui/src/utils/addonsClient.js | 6 + core/http/react-ui/src/utils/capabilities.js | 1 + core/http/react-ui/src/utils/chatAddons.js | 47 + core/http/react-ui/src/utils/destinations.js | 24 +- core/http/react-ui/src/utils/legacyRoutes.js | 66 + core/http/react-ui/src/utils/modalLayers.js | 17 + .../react-ui/src/utils/navigationModel.js | 20 + core/http/react-ui/src/utils/pinsApi.js | 31 + core/http/react-ui/src/utils/pinsClient.js | 5 + .../http/react-ui/src/utils/settingsSchema.js | 2 + core/http/react-ui/tests/unit/addons.test.mjs | 280 + .../react-ui/tests/unit/chataddons.test.mjs | 62 + core/http/react-ui/tests/unit/esm.test.mjs | 689 ++ .../react-ui/tests/unit/legacyroutes.test.mjs | 125 + .../react-ui/tests/unit/navigation.test.mjs | 187 + .../react-ui/tests/unit/recovery.test.mjs | 114 + .../react-ui/tests/unit/resources.test.mjs | 202 + core/http/react-ui/tests/unit/run.mjs | 38 + core/http/react-ui/tests/unit/sdkv2.test.mjs | 184 + core/http/react-ui/tests/unit/slots.test.mjs | 296 + .../tests/unit/tablesartifacts.test.mjs | 85 + .../react-ui/tests/unit/tablestools.test.mjs | 105 + .../http/react-ui/tests/unit/updates.test.mjs | 108 + core/http/routes/addon_navigation.go | 39 + core/http/routes/addon_navigation_test.go | 191 + core/http/routes/addons.go | 97 + core/http/routes/addons_discovery_test.go | 107 + core/http/routes/localai.go | 3 + core/http/routes/openai.go | 2 +- core/schema/chat_conversation.go | 2 + core/services/addons/apps/apps.go | 293 + core/services/addons/apps/apps_test.go | 175 + core/services/addons/apps/installed.go | 121 + core/services/addons/apps_test.go | 289 + core/services/addons/artifacts/artifacts.go | 398 + .../addons/artifacts/artifacts_test.go | 1895 +++++ core/services/addons/artifacts/expiry.go | 249 + .../services/addons/artifacts/refusal_test.go | 38 + core/services/addons/artifacts/safe.go | 107 + core/services/addons/artifacts/store.go | 485 ++ core/services/addons/artifacts/transfer.go | 598 ++ core/services/addons/autoupgrade.go | 100 + core/services/addons/autoupgrade_test.go | 140 + core/services/addons/builtin/builtin.go | 82 + .../addons/builtin/builtin_suite_test.go | 14 + core/services/addons/builtin/builtin_test.go | 59 + core/services/addons/catalog/catalog.go | 241 + core/services/addons/catalog/catalog_test.go | 327 + core/services/addons/catalog/credentials.go | 50 + core/services/addons/catalog/index.go | 119 + .../addons/catalog/index_yaml_test.go | 113 + core/services/addons/catalog/official.go | 28 + core/services/addons/catalog/official_test.go | 83 + core/services/addons/catalog/sources.go | 423 ++ core/services/addons/catalog/sources_test.go | 241 + core/services/addons/gateway/apps.go | 106 + core/services/addons/gateway/artifacts.go | 142 + .../services/addons/gateway/artifacts_test.go | 76 + core/services/addons/gateway/carrier.go | 599 ++ core/services/addons/gateway/carrier_test.go | 199 + core/services/addons/gateway/forwarding.go | 116 + core/services/addons/gateway/gateway.go | 138 + core/services/addons/gateway/gateway_test.go | 209 + .../services/addons/gateway/hardening_test.go | 284 + .../services/addons/gateway/lifecycle_test.go | 56 + core/services/addons/gateway/limits.go | 28 + core/services/addons/gateway/loopback.go | 151 + core/services/addons/gateway/loopback_test.go | 75 + core/services/addons/gateway/management.go | 89 + core/services/addons/gateway/mutation_test.go | 24 + .../addons/gateway/ownerfixture_test.go | 76 + core/services/addons/gateway/proposals.go | 551 ++ core/services/addons/gateway/recovery.go | 188 + core/services/addons/gateway/refusal_test.go | 99 + core/services/addons/gateway/relay.go | 414 + core/services/addons/gateway/relay_test.go | 256 + .../addons/gateway/resourceless_test.go | 114 + core/services/addons/gateway/resources.go | 614 ++ core/services/addons/gateway/status_test.go | 241 + core/services/addons/gateway/tools.go | 200 + core/services/addons/gateway/tools_test.go | 216 + .../services/addons/gateway/transport_test.go | 155 + core/services/addons/gateway/updates.go | 133 + core/services/addons/hostops/broker.go | 301 + core/services/addons/hostops/broker_test.go | 156 + .../addons/hostops/engine_datasets.go | 127 + .../addons/hostops/engine_datasets_test.go | 159 + core/services/addons/hostops/engine_jobs.go | 350 + .../addons/hostops/engine_jobs_test.go | 373 + core/services/addons/hostops/engine_models.go | 154 + .../addons/hostops/engine_models_test.go | 269 + .../addons/hostops/engine_progress.go | 224 + .../addons/hostops/engine_progress_test.go | 200 + core/services/addons/hostops/inflight.go | 68 + core/services/addons/hostops/registry_test.go | 84 + core/services/addons/hostops/suite_test.go | 14 + core/services/addons/journal.go | 93 + core/services/addons/management.go | 202 + core/services/addons/manager.go | 1025 +++ core/services/addons/manager_test.go | 401 + core/services/addons/ownership.go | 81 + core/services/addons/packages/extract.go | 128 + core/services/addons/packages/icon_test.go | 111 + core/services/addons/packages/install.go | 389 + .../addons/packages/install_redirect_test.go | 131 + core/services/addons/packages/install_test.go | 604 ++ core/services/addons/packages/receipt.go | 110 + core/services/addons/pins/pins.go | 60 + core/services/addons/pins/service.go | 64 + .../services/addons/pins/service_gorm_test.go | 46 + core/services/addons/pins/service_test.go | 122 + .../addons/pins/store_contract_test.go | 120 + core/services/addons/pins/store_file.go | 186 + core/services/addons/pins/store_file_test.go | 61 + core/services/addons/pins/store_gorm.go | 113 + core/services/addons/pins/store_gorm_test.go | 54 + core/services/addons/pins/suite_test.go | 14 + core/services/addons/pins/sweep.go | 37 + core/services/addons/pins/sweep_test.go | 55 + core/services/addons/policy/acl.go | 101 + core/services/addons/policy/addon_state.go | 514 ++ .../addons/policy/addon_state_test.go | 400 + core/services/addons/policy/admission.go | 758 ++ .../addons/policy/admit_addon_test.go | 78 + core/services/addons/policy/backing_test.go | 94 + core/services/addons/policy/broker_manage.go | 46 + core/services/addons/policy/creates_test.go | 574 ++ core/services/addons/policy/identity.go | 65 + core/services/addons/policy/manage.go | 375 + core/services/addons/policy/policy_test.go | 1110 +++ core/services/addons/policy/proposals.go | 1531 ++++ .../addons/policy/proposals_kind_test.go | 110 + .../addons/policy/proposals_model_test.go | 81 + .../addons/policy/proposals_notsent_test.go | 111 + core/services/addons/policy/proposals_test.go | 1529 ++++ .../addons/policy/reconcile_soft_test.go | 102 + core/services/addons/policy/recovery.go | 443 ++ core/services/addons/policy/refusal_test.go | 97 + core/services/addons/policy/resources.go | 346 + core/services/addons/policy/store.go | 516 ++ core/services/addons/policy/tools_test.go | 169 + core/services/addons/recovery.go | 277 + core/services/addons/recovery/backup.go | 823 ++ core/services/addons/recovery/backup_test.go | 228 + core/services/addons/recovery/export_test.go | 13 + core/services/addons/recovery/files.go | 306 + core/services/addons/recovery/migration.go | 296 + core/services/addons/recovery/purge.go | 319 + core/services/addons/recovery/purge_test.go | 111 + .../addons/recovery/recovery_suite_test.go | 14 + core/services/addons/recovery/restore.go | 506 ++ core/services/addons/recovery/restore_test.go | 220 + core/services/addons/recovery/rig_test.go | 362 + core/services/addons/recovery/scope_test.go | 96 + core/services/addons/recovery/state.go | 291 + core/services/addons/recovery/types.go | 195 + core/services/addons/recovery/upgrade_test.go | 248 + .../addons/resources/delete_backing_test.go | 129 + .../addons/resources/list_budget_test.go | 77 + .../services/addons/resources/refusal_test.go | 83 + core/services/addons/resources/selection.go | 327 + core/services/addons/resources/service.go | 1446 ++++ core/services/addons/runtime/client.go | 330 + core/services/addons/runtime/client_test.go | 270 + core/services/addons/runtime/ledger_linux.go | 228 + core/services/addons/runtime/ledger_other.go | 21 + core/services/addons/runtime/supervisor.go | 749 ++ .../addons/runtime/supervisor_test.go | 709 ++ .../addons/runtime_integration_test.go | 242 + core/services/addons/state.go | 39 + core/services/addons/storage.go | 185 + core/services/addons/storage_test.go | 153 + core/services/addons/toolargs/toolargs.go | 196 + .../addons/toolargs/toolargs_suite_test.go | 14 + .../services/addons/toolargs/toolargs_test.go | 33 + core/services/addons/tools/adapter.go | 421 ++ core/services/addons/tools/adapter_test.go | 437 ++ .../addons/tools/consumer_matrix_test.go | 193 + core/services/addons/tools/consumers.go | 148 + .../services/addons/tools/definitions_test.go | 135 + core/services/addons/tools/executor_test.go | 284 + .../services/addons/tools/tools_suite_test.go | 14 + core/services/addons/tools_test.go | 69 + core/services/addons/updates.go | 175 + core/services/addons/updates/checker.go | 561 ++ core/services/addons/updates/checker_test.go | 514 ++ core/services/addons/updates/semver.go | 89 + core/services/addons/updates/status.go | 107 + core/services/addons/updates_test.go | 494 ++ core/services/advisorylock/keys.go | 21 +- core/services/agentpool/agent_jobs.go | 9 +- core/services/agents/toolpolicy.go | 12 +- core/services/agents/toolpolicy_test.go | 11 + core/services/finetune/export_name_test.go | 52 + core/services/finetune/service.go | 15 +- .../services/quantization/import_name_test.go | 52 + core/services/quantization/service.go | 17 +- docs/content/development/addon-testing.md | 387 + docs/content/features/addons.md | 2750 +++++++ docs/content/features/face-recognition.md | 2 +- docs/content/features/fine-tuning.md | 10 + docs/content/features/quantization.md | 10 + docs/content/features/studio.md | 4 + docs/content/features/tables.md | 765 ++ docs/content/features/text-to-audio.md | 2 +- docs/content/features/voice-recognition.md | 4 +- docs/content/reference/cli-reference.md | 1 + .../plans/2026-10-16-addons-navigation.md | 6649 +++++++++++++++++ .../2026-10-16-addons-navigation-design.md | 534 ++ gallery/addons/README.md | 65 + gallery/addons/face.yaml.tmpl | 18 + gallery/addons/fine-tuning.yaml.tmpl | 18 + gallery/addons/index.yaml | 131 + gallery/addons/quantization.yaml.tmpl | 18 + gallery/addons/studio.yaml.tmpl | 18 + gallery/addons/tables.yaml.tmpl | 18 + gallery/addons/talk.yaml.tmpl | 18 + gallery/addons/voice.yaml.tmpl | 18 + pkg/addons/contract/contract_suite_test.go | 10 + pkg/addons/contract/contributions.go | 214 + pkg/addons/contract/hostops.go | 70 + pkg/addons/contract/hostops_test.go | 71 + pkg/addons/contract/limits.go | 49 + pkg/addons/contract/manifest.go | 506 ++ pkg/addons/contract/manifest_test.go | 355 + pkg/addons/contract/protocol.go | 223 + pkg/addons/contract/protocol_test.go | 103 + pkg/addons/contract/tooldefs.go | 272 + pkg/addons/contract/tooldefs_test.go | 37 + pkg/addons/model/proposalflow/README.md | 209 + .../model/proposalflow/cmd/viewer/main.go | 117 + .../proposalflow/cmd/viewer/viewer.tmpl.html | 99 + pkg/addons/model/proposalflow/explore.go | 445 ++ pkg/addons/model/proposalflow/explore_test.go | 63 + pkg/addons/model/proposalflow/gen.go | 9 + pkg/addons/model/proposalflow/reduce.go | 1391 ++++ pkg/addons/model/proposalflow/reduce_test.go | 130 + pkg/addons/model/proposalflow/suite_test.go | 15 + pkg/addons/model/proposalflow/traces.go | 514 ++ pkg/addons/model/proposalflow/traces_test.go | 85 + pkg/addons/model/proposalflow/viewer.html | 99 + pkg/addons/packaging/build.go | 311 + pkg/addons/packaging/catalog.go | 243 + pkg/addons/packaging/channel.go | 174 + pkg/addons/packaging/config.go | 150 + pkg/addons/packaging/helpers_test.go | 342 + pkg/addons/packaging/layout.go | 186 + pkg/addons/packaging/merge.go | 349 + pkg/addons/packaging/packaging_test.go | 1321 ++++ pkg/addons/packaging/platforms_test.go | 77 + pkg/addons/packaging/policy.go | 390 + pkg/addons/packaging/registry.go | 528 ++ pkg/addons/packaging/run.go | 47 + pkg/addons/packaging/verify.go | 373 + pkg/oci/cosignverify/offline_internal_test.go | 67 + pkg/oci/cosignverify/verify.go | 33 +- pkg/utils/model_name.go | 19 + pkg/utils/model_name_test.go | 51 + scripts/addons/channel.sh | 19 + scripts/addons/cmd/addonpkg/main.go | 20 + scripts/addons/esm_ui.sh | 80 + scripts/addons/foundation.sh | 212 + scripts/addons/foundation_gate.sh | 66 + scripts/addons/gallery_ui.sh | 41 + scripts/addons/host_slots.sh | 72 + scripts/addons/leaks.sh | 48 + scripts/addons/package.sh | 82 + scripts/addons/package_test.sh | 172 + scripts/addons/prepare.sh | 84 + scripts/addons/prerequisites_test.sh | 210 + scripts/addons/recovery.sh | 104 + scripts/addons/suites.json | 3455 +++++++++ scripts/addons/tables_acceptance.sh | 170 + scripts/addons/tables_csv.sh | 113 + scripts/addons/tables_gate.sh | 65 + scripts/addons/tables_tools.sh | 95 + scripts/addons/tables_ui.sh | 110 + scripts/addons/ui_cases.py | 58 + scripts/addons/updates.sh | 121 + scripts/addons/verify.sh | 156 + swagger/docs.go | 4356 +++++++++++ swagger/swagger.json | 4356 +++++++++++ swagger/swagger.yaml | 3539 +++++++++ tests/addons/fixture/cmd/main.go | 244 + tests/addons/fixture/ui/bare/manifest.json | 1 + tests/addons/fixture/ui/bare/ui.js | 7 + tests/addons/fixture/ui/cssb/manifest.json | 1 + tests/addons/fixture/ui/cssb/style.css | 2 + tests/addons/fixture/ui/cssb/ui.js | 8 + tests/addons/fixture/ui/slots/manifest.json | 1 + tests/addons/fixture/ui/slots/ui.js | 67 + tests/addons/fixture/ui/slots/ui/style.css | 10 + .../fixture/ui/uifixture-2/manifest.json | 1 + tests/addons/fixture/ui/uifixture-2/ui.js | 11 + .../fixture/ui/uifixture-2/ui/style.css | 9 + .../addons/fixture/ui/uifixture/manifest.json | 1 + tests/addons/fixture/ui/uifixture/ui.js | 95 + .../addons/fixture/ui/uifixture/ui/style.css | 9 + tests/addons/fixtures/contracts/app.json | 55 + tests/addons/fixtures/contracts/chat.json | 36 + tests/addons/fixtures/contracts/envelope.json | 1 + tests/addons/fixtures/contracts/headless.json | 30 + .../fixtures/contracts/incompatible.json | 65 + tests/addons/fixtures/contracts/page.json | 42 + tests/addons/fixtures/contracts/service.json | 1 + tests/addons/fixtures/packages/manifest.json | 1 + tests/addons/fixtures/signing/signing.go | 158 + tests/addons/foundation/cluster/catalog.go | 187 + tests/addons/foundation/cluster/cluster.go | 107 + tests/addons/foundation/cluster/config.go | 292 + tests/addons/foundation/cluster/nodes.go | 354 + tests/addons/foundation/cluster/packages.go | 42 + tests/addons/foundation/cluster/server.go | 182 + .../foundation/cluster/teardown_test.go | 98 + .../foundation/cmd/foundation-cluster/main.go | 98 + .../foundation/foundation_suite_test.go | 188 + tests/addons/foundation/foundation_test.go | 506 ++ .../foundation/recovery_failure_test.go | 129 + tests/addons/foundation/recovery_test.go | 186 + tests/addons/foundation/service_test.go | 279 + tests/addons/foundation/two_frontends_test.go | 102 + .../addons/hostslots/hostslots_suite_test.go | 14 + tests/addons/hostslots/hostslots_test.go | 191 + tests/addons/packaging/packaging_test.go | 300 + tests/addons/recovery/host_test.go | 475 ++ tests/addons/recovery/recovery_test.go | 463 ++ tests/addons/tables/application_test.go | 393 + tests/addons/tables/databases_test.go | 270 + tests/addons/tables/rows_test.go | 284 + tests/addons/tables/schema_test.go | 182 + tests/addons/tables/tables_suite_test.go | 159 + tests/addons/tables/tools_test.go | 240 + tests/addons/tablesacceptance/.gitignore | 1 + .../tablesacceptance/authorization_test.go | 437 ++ .../cmd/tables-cluster/main.go | 157 + .../tablesacceptance/concurrency_test.go | 265 + .../addons/tablesacceptance/evidence_test.go | 159 + tests/addons/tablesacceptance/faults_test.go | 211 + tests/addons/tablesacceptance/helpers_test.go | 347 + .../addons/tablesacceptance/recovery_test.go | 357 + .../addons/tablesacceptance/refusals_test.go | 81 + .../addons/tablesacceptance/replicas_test.go | 464 ++ .../tablesacceptance/tables_suite_test.go | 287 + .../tablesacceptance/tablespkg/tablespkg.go | 156 + tests/addons/tablesadmin/apply_test.go | 270 + tests/addons/tablesadmin/delete_test.go | 307 + tests/addons/tablesadmin/docs_test.go | 144 + tests/addons/tablesadmin/export_test.go | 455 ++ tests/addons/tablesadmin/host_test.go | 485 ++ tests/addons/tablesadmin/parity_test.go | 161 + tests/addons/tablesadmin/resources_test.go | 286 + tests/addons/tablesadmin/selection_test.go | 278 + tests/addons/tablesadmin/store_test.go | 314 + .../addons/tablesexport/export_suite_test.go | 81 + tests/addons/tablesexport/export_test.go | 209 + tests/e2e-ui/addons_fixture.go | 454 ++ tests/e2e-ui/main.go | 96 +- tests/e2e/mock-backend/main.go | 76 +- 964 files changed, 159560 insertions(+), 515 deletions(-) create mode 100644 .github/workflows/addons-package.yml create mode 100644 addons/face/cmd/face/main.go create mode 100644 addons/face/manifest.json create mode 100644 addons/face/package/platforms.yaml create mode 100644 addons/face/ui/.gitignore create mode 100644 addons/face/ui/build.mjs create mode 100644 addons/face/ui/icon.png create mode 100644 addons/face/ui/package.json create mode 100644 addons/face/ui/src/FaceRecognition.jsx create mode 100644 addons/face/ui/src/MoreFaceTools.jsx create mode 100644 addons/face/ui/src/api.js create mode 100644 addons/face/ui/src/ctx.jsx create mode 100644 addons/face/ui/src/i18n/en.js create mode 100644 addons/face/ui/src/index.jsx create mode 100644 addons/face/ui/src/lib/model.js create mode 100644 addons/face/ui/src/style.css create mode 100644 addons/face/ui/test/model.test.mjs create mode 100644 addons/face/ui/test/recognition.test.mjs create mode 100644 addons/face/ui/test/render.test.mjs create mode 100644 addons/face/ui/test/run.mjs create mode 100644 addons/fine-tuning/cmd/fine-tuning/main.go create mode 100644 addons/fine-tuning/internal/tools/journal.go create mode 100644 addons/fine-tuning/internal/tools/suite_test.go create mode 100644 addons/fine-tuning/internal/tools/tools.go create mode 100644 addons/fine-tuning/internal/tools/tools_test.go create mode 100644 addons/fine-tuning/manifest.json create mode 100644 addons/fine-tuning/package/platforms.yaml create mode 100644 addons/fine-tuning/ui/.gitignore create mode 100644 addons/fine-tuning/ui/build.mjs create mode 100644 addons/fine-tuning/ui/icon.png create mode 100644 addons/fine-tuning/ui/package.json create mode 100644 addons/fine-tuning/ui/src/FineTune.jsx create mode 100644 addons/fine-tuning/ui/src/LossChart.jsx create mode 100644 addons/fine-tuning/ui/src/api.js create mode 100644 addons/fine-tuning/ui/src/ctx.jsx create mode 100644 addons/fine-tuning/ui/src/i18n/en.js create mode 100644 addons/fine-tuning/ui/src/index.jsx create mode 100644 addons/fine-tuning/ui/src/lib/model.js create mode 100644 addons/fine-tuning/ui/src/lib/progress.js create mode 100644 addons/fine-tuning/ui/src/style.css create mode 100644 addons/fine-tuning/ui/test/load.mjs create mode 100644 addons/fine-tuning/ui/test/model.test.mjs create mode 100644 addons/fine-tuning/ui/test/page.test.mjs create mode 100644 addons/fine-tuning/ui/test/progress.test.mjs create mode 100644 addons/fine-tuning/ui/test/render.test.mjs create mode 100644 addons/fine-tuning/ui/test/run.mjs create mode 100644 addons/internal/host-compat/package.json create mode 100644 addons/internal/host-compat/src/components/ConfirmDialog.jsx create mode 100644 addons/internal/host-compat/src/components/Icon.jsx create mode 100644 addons/internal/host-compat/src/components/Lightbox.jsx create mode 100644 addons/internal/host-compat/src/components/LoadingSpinner.jsx create mode 100644 addons/internal/host-compat/src/components/Modal.jsx create mode 100644 addons/internal/host-compat/src/components/ModelSelector.jsx create mode 100644 addons/internal/host-compat/src/components/PageHeader.jsx create mode 100644 addons/internal/host-compat/src/components/audio/WaveformPlayer.jsx create mode 100644 addons/internal/host-compat/src/components/biometrics/MediaInput.jsx create mode 100644 addons/internal/host-compat/src/components/home/HomeModelPicker.jsx create mode 100644 addons/internal/host-compat/src/components/home/memory.js create mode 100644 addons/internal/host-compat/src/context/AuthContext.jsx create mode 100644 addons/internal/host-compat/src/hooks/useAudioPeaks.js create mode 100644 addons/internal/host-compat/src/hooks/useMCPClient.js create mode 100644 addons/internal/host-compat/src/hooks/useMediaCapture.js create mode 100644 addons/internal/host-compat/src/hooks/useModels.js create mode 100644 addons/internal/host-compat/src/hooks/useObjectUrl.js create mode 100644 addons/internal/host-compat/src/hooks/useOperations.js create mode 100644 addons/internal/host-compat/src/hooks/useResources.js create mode 100644 addons/internal/host-compat/src/hooks/useVoiceProfiles.js create mode 100644 addons/internal/host-compat/src/i18n.js create mode 100644 addons/internal/host-compat/src/modules/jsx-runtime.js create mode 100644 addons/internal/host-compat/src/modules/react-dom.js create mode 100644 addons/internal/host-compat/src/modules/react-i18next.js create mode 100644 addons/internal/host-compat/src/modules/react-router-dom.js create mode 100644 addons/internal/host-compat/src/modules/react.js create mode 100644 addons/internal/host-compat/src/route.js create mode 100644 addons/internal/host-compat/src/runtime.js create mode 100644 addons/internal/host-compat/src/ui.js create mode 100644 addons/internal/host-compat/src/utils/api.js create mode 100644 addons/internal/host-compat/src/utils/basePath.js create mode 100644 addons/internal/host-compat/src/utils/capabilities.js create mode 100644 addons/internal/host-compat/src/utils/clipboard.js create mode 100644 addons/internal/host-compat/src/utils/config.js create mode 100644 addons/internal/host-compat/src/utils/editorNav.js create mode 100644 addons/internal/host-compat/src/utils/format.js create mode 100644 addons/internal/host-compat/src/utils/lastModel.js create mode 100644 addons/internal/host-compat/src/utils/mcpClientStorage.js create mode 100644 addons/internal/host-compat/src/utils/modelBudget.js create mode 100644 addons/internal/host-compat/src/utils/modelLedger.js create mode 100644 addons/internal/host-compat/test/api.test.mjs create mode 100644 addons/internal/host-compat/test/copies.test.mjs create mode 100644 addons/internal/host-compat/test/host.mjs create mode 100644 addons/internal/host-compat/test/i18n.test.mjs create mode 100644 addons/internal/host-compat/test/loader.mjs create mode 100644 addons/internal/host-compat/test/parity.mjs create mode 100644 addons/internal/host-compat/test/register.mjs create mode 100644 addons/internal/host-compat/test/route.test.mjs create mode 100644 addons/internal/host-compat/test/router.test.mjs create mode 100644 addons/internal/host-compat/test/run.mjs create mode 100644 addons/internal/hostclient/client.go create mode 100644 addons/internal/hostclient/client_test.go create mode 100644 addons/internal/hostclient/suite_test.go create mode 100644 addons/internal/identity-ui/src/BoundingBoxCanvas.jsx create mode 100644 addons/internal/identity-ui/src/ClipInput.jsx create mode 100644 addons/internal/identity-ui/src/DistanceScale.jsx create mode 100644 addons/internal/identity-ui/src/EnrolSheet.jsx create mode 100644 addons/internal/identity-ui/src/ErrorNote.jsx create mode 100644 addons/internal/identity-ui/src/KnownList.jsx create mode 100644 addons/internal/identity-ui/src/ModelNeeded.jsx create mode 100644 addons/internal/identity-ui/src/Verdict.jsx create mode 100644 addons/internal/identity-ui/src/Workbench.jsx create mode 100644 addons/internal/identity-ui/src/identity.js create mode 100644 addons/internal/identity-ui/src/referenceAudio.js create mode 100644 addons/internal/identity-ui/src/registry.js create mode 100644 addons/internal/identity-ui/test/identity.test.mjs create mode 100644 addons/internal/idle/idle.go create mode 100644 addons/internal/idle/idle_test.go create mode 100644 addons/internal/idle/suite_test.go create mode 100644 addons/internal/protocol/protocol.go create mode 100644 addons/internal/protocol/run_test.go create mode 100644 addons/internal/protocol/suite_test.go create mode 100644 addons/internal/tools-ui/src/Checks.jsx create mode 100644 addons/internal/tools-ui/src/JobLog.jsx create mode 100644 addons/internal/tools-ui/src/StageLine.jsx create mode 100644 addons/internal/tools-ui/src/ToolOff.jsx create mode 100644 addons/internal/tools-ui/src/ToolSteps.jsx create mode 100644 addons/internal/tools-ui/src/format.js create mode 100644 addons/internal/tools-ui/src/sse.js create mode 100644 addons/internal/tools-ui/src/tools.css create mode 100644 addons/internal/tools-ui/src/tools.js create mode 100644 addons/internal/tools-ui/test/copies.test.mjs create mode 100644 addons/internal/tools-ui/test/fixture/.gitignore create mode 100644 addons/internal/tools-ui/test/fixture/src/index.jsx create mode 100644 addons/internal/tools-ui/test/render.test.mjs create mode 100644 addons/internal/tools-ui/test/sse.test.mjs create mode 100644 addons/internal/tools-ui/test/tools.test.mjs create mode 100644 addons/internal/uibuild/package.json create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/components/Badge.jsx create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/modules/jsx-runtime.js create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/modules/react-dom.js create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/modules/react-i18next.js create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/modules/react-router-dom.js create mode 100644 addons/internal/uibuild/test/fixture-compat/compat/modules/react.js create mode 100644 addons/internal/uibuild/test/fixture-compat/host-compat-fixture-runtime.js create mode 100644 addons/internal/uibuild/test/fixture-compat/src/Page.jsx create mode 100644 addons/internal/uibuild/test/fixture-compat/src/index.jsx create mode 100644 addons/internal/uibuild/test/fixture-compat/src/page.css create mode 100644 addons/internal/uibuild/test/fixture-compat/src/style.css create mode 100644 addons/internal/uibuild/test/fixture-core/src/index.jsx create mode 100644 addons/internal/uibuild/test/fixture-core/src/style.css create mode 100644 addons/internal/uibuild/test/fixture-icon/icon.png create mode 100644 addons/internal/uibuild/test/fixture-icon/src/index.jsx create mode 100644 addons/internal/uibuild/test/fixture-icon/src/style.css create mode 100644 addons/internal/uibuild/test/fixture-react/src/index.jsx create mode 100644 addons/internal/uibuild/test/fixture-react/src/style.css create mode 100644 addons/internal/uibuild/test/fixture/src/index.jsx create mode 100644 addons/internal/uibuild/test/fixture/src/style.css create mode 100644 addons/internal/uibuild/test/uibuild.test.mjs create mode 100644 addons/internal/uibuild/uibuild.mjs create mode 100644 addons/quantization/cmd/quantization/main.go create mode 100644 addons/quantization/internal/tools/journal.go create mode 100644 addons/quantization/internal/tools/suite_test.go create mode 100644 addons/quantization/internal/tools/tools.go create mode 100644 addons/quantization/internal/tools/tools_test.go create mode 100644 addons/quantization/manifest.json create mode 100644 addons/quantization/package/platforms.yaml create mode 100644 addons/quantization/ui/.gitignore create mode 100644 addons/quantization/ui/build.mjs create mode 100644 addons/quantization/ui/icon.png create mode 100644 addons/quantization/ui/package.json create mode 100644 addons/quantization/ui/src/Quantize.jsx create mode 100644 addons/quantization/ui/src/api.js create mode 100644 addons/quantization/ui/src/ctx.jsx create mode 100644 addons/quantization/ui/src/i18n/en.js create mode 100644 addons/quantization/ui/src/index.jsx create mode 100644 addons/quantization/ui/src/lib/model.js create mode 100644 addons/quantization/ui/src/lib/progress.js create mode 100644 addons/quantization/ui/src/style.css create mode 100644 addons/quantization/ui/test/load.mjs create mode 100644 addons/quantization/ui/test/model.test.mjs create mode 100644 addons/quantization/ui/test/page.test.mjs create mode 100644 addons/quantization/ui/test/progress.test.mjs create mode 100644 addons/quantization/ui/test/render.test.mjs create mode 100644 addons/quantization/ui/test/run.mjs create mode 100644 addons/studio/cmd/studio/main.go create mode 100644 addons/studio/manifest.json create mode 100644 addons/studio/package/platforms.yaml create mode 100644 addons/studio/ui/.gitignore create mode 100644 addons/studio/ui/build.mjs create mode 100644 addons/studio/ui/icon.png create mode 100644 addons/studio/ui/package.json create mode 100644 addons/studio/ui/src/addon.jsx rename {core/http/react-ui => addons/studio/ui}/src/components/AnimationOptions.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/AnimationViewer.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/GlbViewer.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/RequestPanel.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/audio/Spectrogram.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/HandoffNote.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/InstallNote.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/LineageView.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/StarButton.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/StudioComposer.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/Timeline.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/UpscaleComposer.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/WorkMasonry.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/WorkThumb.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/Workspace.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/studio.css (100%) rename {core/http/react-ui => addons/studio/ui}/src/components/studio/workspace.css (98%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/use3DHistory.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useMediaHistory.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useModelSuggestion.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useSoundExpansionModel.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useSpectrogram.js (97%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useStudioHandoff.js (95%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useStudioWork.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/hooks/useWorkspace.js (100%) create mode 100644 addons/studio/ui/src/i18n/media.de.json create mode 100644 addons/studio/ui/src/i18n/media.en.json create mode 100644 addons/studio/ui/src/i18n/media.es.json create mode 100644 addons/studio/ui/src/i18n/media.id.json create mode 100644 addons/studio/ui/src/i18n/media.it.json create mode 100644 addons/studio/ui/src/i18n/media.ko.json create mode 100644 addons/studio/ui/src/i18n/media.pt-BR.json create mode 100644 addons/studio/ui/src/i18n/media.zh-CN.json create mode 100644 addons/studio/ui/src/index.jsx rename {core/http/react-ui => addons/studio/ui}/src/pages/AudioTransform.jsx (99%) rename {core/http/react-ui => addons/studio/ui}/src/pages/Diarization.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/ImageGen.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/Sound.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/Studio.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/StudioOverview.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/TTS.jsx (100%) rename {core/http/react-ui => addons/studio/ui}/src/pages/ThreeDGen.jsx (99%) rename {core/http/react-ui => addons/studio/ui}/src/pages/VideoGen.jsx (100%) create mode 100644 addons/studio/ui/src/style.css rename {core/http/react-ui => addons/studio/ui}/src/utils/animationGlb.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/diarization.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/fft.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/glb.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/mediaHistoryStore.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/studioWork.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/upscale.js (100%) rename {core/http/react-ui => addons/studio/ui}/src/utils/voiceEnrollments.js (100%) rename core/http/react-ui/src/utils/diarization.test.js => addons/studio/ui/test/diarization.test.mjs (98%) create mode 100644 addons/studio/ui/test/parity.test.mjs create mode 100644 addons/studio/ui/test/render.test.mjs create mode 100644 addons/studio/ui/test/run.mjs rename core/http/react-ui/src/utils/studioWork.test.js => addons/studio/ui/test/studioWork.test.mjs (99%) rename core/http/react-ui/src/utils/upscale.test.js => addons/studio/ui/test/upscale.test.mjs (87%) create mode 100644 addons/tables/cmd/tables-faults/main.go create mode 100644 addons/tables/cmd/tables/main.go create mode 100644 addons/tables/internal/server/export.go create mode 100644 addons/tables/internal/server/export_test.go create mode 100644 addons/tables/internal/server/rows.go create mode 100644 addons/tables/internal/server/run.go create mode 100644 addons/tables/internal/server/schema_authority_test.go create mode 100644 addons/tables/internal/server/server_suite_test.go create mode 100644 addons/tables/internal/server/tables.go create mode 100644 addons/tables/internal/store/capacity_internal_test.go create mode 100644 addons/tables/internal/store/databases.go create mode 100644 addons/tables/internal/store/databases_test.go create mode 100644 addons/tables/internal/store/export.go create mode 100644 addons/tables/internal/store/export_test.go create mode 100644 addons/tables/internal/store/migrate.go create mode 100644 addons/tables/internal/store/migrate_internal_test.go create mode 100644 addons/tables/internal/store/migrate_test.go create mode 100644 addons/tables/internal/store/query.go create mode 100644 addons/tables/internal/store/query_test.go create mode 100644 addons/tables/internal/store/receipts.go create mode 100644 addons/tables/internal/store/receipts_test.go create mode 100644 addons/tables/internal/store/revisions.go create mode 100644 addons/tables/internal/store/rows.go create mode 100644 addons/tables/internal/store/rows_test.go create mode 100644 addons/tables/internal/store/schema.go create mode 100644 addons/tables/internal/store/schema_test.go create mode 100644 addons/tables/internal/store/store_suite_test.go create mode 100644 addons/tables/manifest.json create mode 100644 addons/tables/package/platforms.yaml create mode 100644 addons/tables/ui/.gitignore create mode 100644 addons/tables/ui/build.mjs create mode 100644 addons/tables/ui/icon.png create mode 100644 addons/tables/ui/package.json create mode 100644 addons/tables/ui/src/Database.jsx create mode 100644 addons/tables/ui/src/Databases.jsx create mode 100644 addons/tables/ui/src/Export.jsx create mode 100644 addons/tables/ui/src/Filters.jsx create mode 100644 addons/tables/ui/src/Rows.jsx create mode 100644 addons/tables/ui/src/Schema.jsx create mode 100644 addons/tables/ui/src/SelectDatabase.jsx create mode 100644 addons/tables/ui/src/Sharing.jsx create mode 100644 addons/tables/ui/src/TableView.jsx create mode 100644 addons/tables/ui/src/api.js create mode 100644 addons/tables/ui/src/ctx.js create mode 100644 addons/tables/ui/src/fields.jsx create mode 100644 addons/tables/ui/src/hooks.js create mode 100644 addons/tables/ui/src/i18n/en.js create mode 100644 addons/tables/ui/src/i18n/index.js create mode 100644 addons/tables/ui/src/i18n/locales.js create mode 100644 addons/tables/ui/src/index.jsx create mode 100644 addons/tables/ui/src/lib/batch.js create mode 100644 addons/tables/ui/src/lib/codecs.js create mode 100644 addons/tables/ui/src/lib/errors.js create mode 100644 addons/tables/ui/src/lib/export.js create mode 100644 addons/tables/ui/src/lib/filters.js create mode 100644 addons/tables/ui/src/lib/query.js create mode 100644 addons/tables/ui/src/lib/retry.js create mode 100644 addons/tables/ui/src/style.css create mode 100644 addons/tables/ui/src/ui.jsx create mode 100644 addons/tables/ui/test/batch.test.mjs create mode 100644 addons/tables/ui/test/codecs.test.mjs create mode 100644 addons/tables/ui/test/filters.test.mjs create mode 100644 addons/tables/ui/test/i18n.test.mjs create mode 100644 addons/tables/ui/test/lint.test.mjs create mode 100644 addons/tables/ui/test/render.test.mjs create mode 100644 addons/tables/ui/test/retry.test.mjs create mode 100644 addons/tables/ui/test/run.mjs create mode 100644 addons/talk/cmd/talk/main.go create mode 100644 addons/talk/manifest.json create mode 100644 addons/talk/package/platforms.yaml create mode 100644 addons/talk/ui/.gitignore create mode 100644 addons/talk/ui/build.mjs create mode 100644 addons/talk/ui/icon.png create mode 100644 addons/talk/ui/package.json create mode 100644 addons/talk/ui/src/addon.jsx rename {core/http/react-ui => addons/talk/ui}/src/components/ClientMCPDropdown.jsx (100%) rename {core/http/react-ui => addons/talk/ui}/src/components/VoiceVisualizer.jsx (100%) rename {core/http/react-ui => addons/talk/ui}/src/components/talk/SessionSheet.jsx (100%) rename core/http/react-ui/public/locales/en/talk.json => addons/talk/ui/src/i18n/talk.en.json (100%) create mode 100644 addons/talk/ui/src/index.jsx rename {core/http/react-ui => addons/talk/ui}/src/pages/Talk.jsx (99%) rename {core/http/react-ui => addons/talk/ui}/src/pages/talk.css (99%) create mode 100644 addons/talk/ui/src/style.css create mode 100644 addons/talk/ui/test/parity.test.mjs create mode 100644 addons/talk/ui/test/render.test.mjs create mode 100644 addons/talk/ui/test/run.mjs create mode 100644 addons/voice/cmd/voice/main.go create mode 100644 addons/voice/manifest.json create mode 100644 addons/voice/package/platforms.yaml create mode 100644 addons/voice/ui/.gitignore create mode 100644 addons/voice/ui/build.mjs create mode 100644 addons/voice/ui/icon.png create mode 100644 addons/voice/ui/package.json create mode 100644 addons/voice/ui/src/MoreVoiceTools.jsx create mode 100644 addons/voice/ui/src/Speech.jsx create mode 100644 addons/voice/ui/src/VoiceHub.jsx create mode 100644 addons/voice/ui/src/VoiceLibrary.jsx create mode 100644 addons/voice/ui/src/VoiceProfileCreate.jsx create mode 100644 addons/voice/ui/src/VoiceRecognition.jsx create mode 100644 addons/voice/ui/src/WaveformStrip.jsx create mode 100644 addons/voice/ui/src/api.js create mode 100644 addons/voice/ui/src/ctx.jsx create mode 100644 addons/voice/ui/src/i18n/en.js create mode 100644 addons/voice/ui/src/index.jsx create mode 100644 addons/voice/ui/src/lib/clipboard.js create mode 100644 addons/voice/ui/src/lib/model.js create mode 100644 addons/voice/ui/src/speechRequest.js create mode 100644 addons/voice/ui/src/style.css create mode 100644 addons/voice/ui/test/library.test.mjs create mode 100644 addons/voice/ui/test/model.test.mjs create mode 100644 addons/voice/ui/test/recognition.test.mjs create mode 100644 addons/voice/ui/test/render.test.mjs create mode 100644 addons/voice/ui/test/run.mjs create mode 100644 addons/voice/ui/test/speech.test.mjs create mode 100644 core/application/addon_admin_auth_test.go create mode 100644 core/application/addon_admin_test.go create mode 100644 core/application/addon_auto_upgrade.go create mode 100644 core/application/addon_auto_upgrade_test.go create mode 100644 core/application/addon_engine_jobs_test.go create mode 100644 core/application/addon_gateway.go create mode 100644 core/application/addon_host_ops_auth_test.go create mode 100644 core/application/addon_host_ops_test.go create mode 100644 core/application/addon_pin_store_internal_test.go create mode 100644 core/application/addon_pin_sweep_test.go create mode 100644 core/application/addon_pins.go create mode 100644 core/application/addon_pins_test.go create mode 100644 core/application/addons_auth_test.go create mode 100644 core/application/addons_other_test.go create mode 100644 core/application/addons_test.go create mode 100644 core/application/startup_auth_test.go create mode 100644 core/cli/agent_worker_mcpfilter_test.go create mode 100644 core/cli/auto_upgrade_addons_test.go create mode 100644 core/cli/run_addon_host_ops_test.go create mode 100644 core/cli/run_addons.go create mode 100644 core/cli/run_addons_test.go create mode 100644 core/config/addon_host_ops.go create mode 100644 core/config/addon_host_ops_test.go create mode 100644 core/http/auth/addons.go create mode 100644 core/http/auth/addons_test.go create mode 100644 core/http/auth/addons_users.go create mode 100644 core/http/auth/core_calls_test.go create mode 100644 core/http/endpoints/addons/apps.go create mode 100644 core/http/endpoints/addons/apps_test.go create mode 100644 core/http/endpoints/addons/artifacts.go create mode 100644 core/http/endpoints/addons/artifacts_test.go create mode 100644 core/http/endpoints/addons/assets.go create mode 100644 core/http/endpoints/addons/assets_test.go create mode 100644 core/http/endpoints/addons/browserguard_test.go create mode 100644 core/http/endpoints/addons/builtin.go create mode 100644 core/http/endpoints/addons/builtin_test.go create mode 100644 core/http/endpoints/addons/catalogui_test.go create mode 100644 core/http/endpoints/addons/forwarding_test.go create mode 100644 core/http/endpoints/addons/gateway.go create mode 100644 core/http/endpoints/addons/gateway_auth_test.go create mode 100644 core/http/endpoints/addons/gateway_test.go create mode 100644 core/http/endpoints/addons/human_surface_test.go create mode 100644 core/http/endpoints/addons/installrefusal_test.go create mode 100644 core/http/endpoints/addons/pins.go create mode 100644 core/http/endpoints/addons/pins_test.go create mode 100644 core/http/endpoints/addons/proposals.go create mode 100644 core/http/endpoints/addons/proposals_test.go create mode 100644 core/http/endpoints/addons/recovery.go create mode 100644 core/http/endpoints/addons/resources.go create mode 100644 core/http/endpoints/addons/resulterror_test.go create mode 100644 core/http/endpoints/addons/routes_test.go create mode 100644 core/http/endpoints/addons/tools_test.go create mode 100644 core/http/endpoints/addons/updates.go create mode 100644 core/http/endpoints/addons/updates_test.go create mode 100644 core/http/endpoints/localai/addon_engine_jobs.go create mode 100644 core/http/endpoints/localai/addon_engine_models_test.go create mode 100644 core/http/endpoints/localai/mcp_addon_tools_internal_test.go create mode 100644 core/http/endpoints/mcp/addon_tools.go create mode 100644 core/http/react-ui/e2e/addon-navigation.spec.js create mode 100644 core/http/react-ui/e2e/addons-fixtures.js create mode 100644 core/http/react-ui/e2e/addons-foundation.spec.js create mode 100644 core/http/react-ui/e2e/addons-gallery.spec.js create mode 100644 core/http/react-ui/e2e/addons-proposals.spec.js create mode 100644 core/http/react-ui/e2e/addons-recovery.spec.js create mode 100644 core/http/react-ui/e2e/addons-runtime.spec.js create mode 100644 core/http/react-ui/e2e/addons-shell.spec.js create mode 100644 core/http/react-ui/e2e/addons-slots.spec.js create mode 100644 core/http/react-ui/e2e/addons-upgrade.spec.js create mode 100644 core/http/react-ui/e2e/chat-addons.spec.js create mode 100644 core/http/react-ui/e2e/global-setup.js create mode 100644 core/http/react-ui/e2e/legacy-redirects.spec.js create mode 100644 core/http/react-ui/e2e/studio-addon.js create mode 100644 core/http/react-ui/e2e/studio-addons.spec.js create mode 100644 core/http/react-ui/e2e/tables-acceptance.spec.js create mode 100644 core/http/react-ui/e2e/tables-artifacts.spec.js create mode 100644 core/http/react-ui/e2e/tables-ui.spec.js create mode 100644 core/http/react-ui/playwright.addons.config.js create mode 100644 core/http/react-ui/playwright.csv.config.js create mode 100644 core/http/react-ui/playwright.esm.config.js create mode 100644 core/http/react-ui/playwright.foundation.config.js create mode 100644 core/http/react-ui/playwright.recovery.config.js create mode 100644 core/http/react-ui/playwright.slots.config.js create mode 100644 core/http/react-ui/playwright.tables-acceptance.config.js create mode 100644 core/http/react-ui/playwright.tables.config.js create mode 100644 core/http/react-ui/playwright.tools.config.js create mode 100644 core/http/react-ui/playwright.updates.config.js create mode 100644 core/http/react-ui/public/locales/de/addons.json create mode 100644 core/http/react-ui/public/locales/en/addons.json create mode 100644 core/http/react-ui/public/locales/es/addons.json create mode 100644 core/http/react-ui/public/locales/fr/addons.json create mode 100644 core/http/react-ui/public/locales/id/addons.json create mode 100644 core/http/react-ui/public/locales/it/addons.json create mode 100644 core/http/react-ui/public/locales/ko/addons.json create mode 100644 core/http/react-ui/public/locales/pt-BR/addons.json create mode 100644 core/http/react-ui/public/locales/zh-CN/addons.json create mode 100644 core/http/react-ui/src/addons/AddonErrorBoundary.jsx create mode 100644 core/http/react-ui/src/addons/AddonRoute.jsx create mode 100644 core/http/react-ui/src/addons/AddonSlot.jsx create mode 100644 core/http/react-ui/src/addons/ApprovalFrame.jsx create mode 100644 core/http/react-ui/src/addons/ArtifactCard.jsx create mode 100644 core/http/react-ui/src/addons/ProposalCard.jsx create mode 100644 core/http/react-ui/src/addons/SlotBoundary.jsx create mode 100644 core/http/react-ui/src/addons/approval.js create mode 100644 core/http/react-ui/src/addons/artifactFacts.js create mode 100644 core/http/react-ui/src/addons/cssScope.js create mode 100644 core/http/react-ui/src/addons/hostComponents.jsx create mode 100644 core/http/react-ui/src/addons/hostServices.js create mode 100644 core/http/react-ui/src/addons/loader.js create mode 100644 core/http/react-ui/src/addons/proposalResult.js create mode 100644 core/http/react-ui/src/addons/registry.jsx create mode 100644 core/http/react-ui/src/addons/registryContext.js create mode 100644 core/http/react-ui/src/addons/registryState.js create mode 100644 core/http/react-ui/src/addons/resourceCalls.js create mode 100644 core/http/react-ui/src/addons/runtime.js create mode 100644 core/http/react-ui/src/addons/sdk.js create mode 100644 core/http/react-ui/src/addons/shell.css create mode 100644 core/http/react-ui/src/addons/slots.js create mode 100644 core/http/react-ui/src/addons/useAddonPage.js create mode 100644 core/http/react-ui/src/addons/useAddonRegistry.js create mode 100644 core/http/react-ui/src/components/AppsPinnedLinks.jsx create mode 100644 core/http/react-ui/src/components/addons/AddonBits.jsx create mode 100644 core/http/react-ui/src/components/addons/AddonTile.jsx create mode 100644 core/http/react-ui/src/components/addons/AppsView.jsx create mode 100644 core/http/react-ui/src/components/addons/BrowseView.jsx create mode 100644 core/http/react-ui/src/components/addons/InstalledView.jsx create mode 100644 core/http/react-ui/src/components/addons/LibrariesView.jsx create mode 100644 core/http/react-ui/src/components/addons/PinControl.jsx create mode 100644 core/http/react-ui/src/components/addons/RecoveryPanel.jsx create mode 100644 core/http/react-ui/src/components/addons/actionBook.js create mode 100644 core/http/react-ui/src/components/addons/addonState.js create mode 100644 core/http/react-ui/src/components/addons/addons.css create mode 100644 core/http/react-ui/src/components/addons/errors.js create mode 100644 core/http/react-ui/src/components/addons/lifecycleFlow.js create mode 100644 core/http/react-ui/src/components/addons/recoveryState.js create mode 100644 core/http/react-ui/src/components/addons/updateState.js create mode 100644 core/http/react-ui/src/components/addons/useAddonActions.js create mode 100644 core/http/react-ui/src/components/catalog/CatalogEmpty.jsx create mode 100644 core/http/react-ui/src/components/catalog/CatalogPager.jsx create mode 100644 core/http/react-ui/src/components/catalog/CatalogSearchBox.jsx create mode 100644 core/http/react-ui/src/components/catalog/CatalogView.jsx create mode 100644 core/http/react-ui/src/components/catalog/SourcesView.jsx create mode 100644 core/http/react-ui/src/components/catalog/catalog.css create mode 100644 core/http/react-ui/src/components/catalog/catalogState.js create mode 100644 core/http/react-ui/src/components/chat/AddonToolsMenu.jsx create mode 100644 core/http/react-ui/src/context/NavigationContext.jsx create mode 100644 core/http/react-ui/src/context/authState.js create mode 100644 core/http/react-ui/src/pages/Addons.jsx create mode 100644 core/http/react-ui/src/utils/addonTiles.js create mode 100644 core/http/react-ui/src/utils/addonsApi.js create mode 100644 core/http/react-ui/src/utils/addonsClient.js create mode 100644 core/http/react-ui/src/utils/chatAddons.js create mode 100644 core/http/react-ui/src/utils/legacyRoutes.js create mode 100644 core/http/react-ui/src/utils/modalLayers.js create mode 100644 core/http/react-ui/src/utils/navigationModel.js create mode 100644 core/http/react-ui/src/utils/pinsApi.js create mode 100644 core/http/react-ui/src/utils/pinsClient.js create mode 100644 core/http/react-ui/tests/unit/addons.test.mjs create mode 100644 core/http/react-ui/tests/unit/chataddons.test.mjs create mode 100644 core/http/react-ui/tests/unit/esm.test.mjs create mode 100644 core/http/react-ui/tests/unit/legacyroutes.test.mjs create mode 100644 core/http/react-ui/tests/unit/navigation.test.mjs create mode 100644 core/http/react-ui/tests/unit/recovery.test.mjs create mode 100644 core/http/react-ui/tests/unit/resources.test.mjs create mode 100644 core/http/react-ui/tests/unit/run.mjs create mode 100644 core/http/react-ui/tests/unit/sdkv2.test.mjs create mode 100644 core/http/react-ui/tests/unit/slots.test.mjs create mode 100644 core/http/react-ui/tests/unit/tablesartifacts.test.mjs create mode 100644 core/http/react-ui/tests/unit/tablestools.test.mjs create mode 100644 core/http/react-ui/tests/unit/updates.test.mjs create mode 100644 core/http/routes/addon_navigation.go create mode 100644 core/http/routes/addon_navigation_test.go create mode 100644 core/http/routes/addons.go create mode 100644 core/http/routes/addons_discovery_test.go create mode 100644 core/services/addons/apps/apps.go create mode 100644 core/services/addons/apps/apps_test.go create mode 100644 core/services/addons/apps/installed.go create mode 100644 core/services/addons/apps_test.go create mode 100644 core/services/addons/artifacts/artifacts.go create mode 100644 core/services/addons/artifacts/artifacts_test.go create mode 100644 core/services/addons/artifacts/expiry.go create mode 100644 core/services/addons/artifacts/refusal_test.go create mode 100644 core/services/addons/artifacts/safe.go create mode 100644 core/services/addons/artifacts/store.go create mode 100644 core/services/addons/artifacts/transfer.go create mode 100644 core/services/addons/autoupgrade.go create mode 100644 core/services/addons/autoupgrade_test.go create mode 100644 core/services/addons/builtin/builtin.go create mode 100644 core/services/addons/builtin/builtin_suite_test.go create mode 100644 core/services/addons/builtin/builtin_test.go create mode 100644 core/services/addons/catalog/catalog.go create mode 100644 core/services/addons/catalog/catalog_test.go create mode 100644 core/services/addons/catalog/credentials.go create mode 100644 core/services/addons/catalog/index.go create mode 100644 core/services/addons/catalog/index_yaml_test.go create mode 100644 core/services/addons/catalog/official.go create mode 100644 core/services/addons/catalog/official_test.go create mode 100644 core/services/addons/catalog/sources.go create mode 100644 core/services/addons/catalog/sources_test.go create mode 100644 core/services/addons/gateway/apps.go create mode 100644 core/services/addons/gateway/artifacts.go create mode 100644 core/services/addons/gateway/artifacts_test.go create mode 100644 core/services/addons/gateway/carrier.go create mode 100644 core/services/addons/gateway/carrier_test.go create mode 100644 core/services/addons/gateway/forwarding.go create mode 100644 core/services/addons/gateway/gateway.go create mode 100644 core/services/addons/gateway/gateway_test.go create mode 100644 core/services/addons/gateway/hardening_test.go create mode 100644 core/services/addons/gateway/lifecycle_test.go create mode 100644 core/services/addons/gateway/limits.go create mode 100644 core/services/addons/gateway/loopback.go create mode 100644 core/services/addons/gateway/loopback_test.go create mode 100644 core/services/addons/gateway/management.go create mode 100644 core/services/addons/gateway/mutation_test.go create mode 100644 core/services/addons/gateway/ownerfixture_test.go create mode 100644 core/services/addons/gateway/proposals.go create mode 100644 core/services/addons/gateway/recovery.go create mode 100644 core/services/addons/gateway/refusal_test.go create mode 100644 core/services/addons/gateway/relay.go create mode 100644 core/services/addons/gateway/relay_test.go create mode 100644 core/services/addons/gateway/resourceless_test.go create mode 100644 core/services/addons/gateway/resources.go create mode 100644 core/services/addons/gateway/status_test.go create mode 100644 core/services/addons/gateway/tools.go create mode 100644 core/services/addons/gateway/tools_test.go create mode 100644 core/services/addons/gateway/transport_test.go create mode 100644 core/services/addons/gateway/updates.go create mode 100644 core/services/addons/hostops/broker.go create mode 100644 core/services/addons/hostops/broker_test.go create mode 100644 core/services/addons/hostops/engine_datasets.go create mode 100644 core/services/addons/hostops/engine_datasets_test.go create mode 100644 core/services/addons/hostops/engine_jobs.go create mode 100644 core/services/addons/hostops/engine_jobs_test.go create mode 100644 core/services/addons/hostops/engine_models.go create mode 100644 core/services/addons/hostops/engine_models_test.go create mode 100644 core/services/addons/hostops/engine_progress.go create mode 100644 core/services/addons/hostops/engine_progress_test.go create mode 100644 core/services/addons/hostops/inflight.go create mode 100644 core/services/addons/hostops/registry_test.go create mode 100644 core/services/addons/hostops/suite_test.go create mode 100644 core/services/addons/journal.go create mode 100644 core/services/addons/management.go create mode 100644 core/services/addons/manager.go create mode 100644 core/services/addons/manager_test.go create mode 100644 core/services/addons/ownership.go create mode 100644 core/services/addons/packages/extract.go create mode 100644 core/services/addons/packages/icon_test.go create mode 100644 core/services/addons/packages/install.go create mode 100644 core/services/addons/packages/install_redirect_test.go create mode 100644 core/services/addons/packages/install_test.go create mode 100644 core/services/addons/packages/receipt.go create mode 100644 core/services/addons/pins/pins.go create mode 100644 core/services/addons/pins/service.go create mode 100644 core/services/addons/pins/service_gorm_test.go create mode 100644 core/services/addons/pins/service_test.go create mode 100644 core/services/addons/pins/store_contract_test.go create mode 100644 core/services/addons/pins/store_file.go create mode 100644 core/services/addons/pins/store_file_test.go create mode 100644 core/services/addons/pins/store_gorm.go create mode 100644 core/services/addons/pins/store_gorm_test.go create mode 100644 core/services/addons/pins/suite_test.go create mode 100644 core/services/addons/pins/sweep.go create mode 100644 core/services/addons/pins/sweep_test.go create mode 100644 core/services/addons/policy/acl.go create mode 100644 core/services/addons/policy/addon_state.go create mode 100644 core/services/addons/policy/addon_state_test.go create mode 100644 core/services/addons/policy/admission.go create mode 100644 core/services/addons/policy/admit_addon_test.go create mode 100644 core/services/addons/policy/backing_test.go create mode 100644 core/services/addons/policy/broker_manage.go create mode 100644 core/services/addons/policy/creates_test.go create mode 100644 core/services/addons/policy/identity.go create mode 100644 core/services/addons/policy/manage.go create mode 100644 core/services/addons/policy/policy_test.go create mode 100644 core/services/addons/policy/proposals.go create mode 100644 core/services/addons/policy/proposals_kind_test.go create mode 100644 core/services/addons/policy/proposals_model_test.go create mode 100644 core/services/addons/policy/proposals_notsent_test.go create mode 100644 core/services/addons/policy/proposals_test.go create mode 100644 core/services/addons/policy/reconcile_soft_test.go create mode 100644 core/services/addons/policy/recovery.go create mode 100644 core/services/addons/policy/refusal_test.go create mode 100644 core/services/addons/policy/resources.go create mode 100644 core/services/addons/policy/store.go create mode 100644 core/services/addons/policy/tools_test.go create mode 100644 core/services/addons/recovery.go create mode 100644 core/services/addons/recovery/backup.go create mode 100644 core/services/addons/recovery/backup_test.go create mode 100644 core/services/addons/recovery/export_test.go create mode 100644 core/services/addons/recovery/files.go create mode 100644 core/services/addons/recovery/migration.go create mode 100644 core/services/addons/recovery/purge.go create mode 100644 core/services/addons/recovery/purge_test.go create mode 100644 core/services/addons/recovery/recovery_suite_test.go create mode 100644 core/services/addons/recovery/restore.go create mode 100644 core/services/addons/recovery/restore_test.go create mode 100644 core/services/addons/recovery/rig_test.go create mode 100644 core/services/addons/recovery/scope_test.go create mode 100644 core/services/addons/recovery/state.go create mode 100644 core/services/addons/recovery/types.go create mode 100644 core/services/addons/recovery/upgrade_test.go create mode 100644 core/services/addons/resources/delete_backing_test.go create mode 100644 core/services/addons/resources/list_budget_test.go create mode 100644 core/services/addons/resources/refusal_test.go create mode 100644 core/services/addons/resources/selection.go create mode 100644 core/services/addons/resources/service.go create mode 100644 core/services/addons/runtime/client.go create mode 100644 core/services/addons/runtime/client_test.go create mode 100644 core/services/addons/runtime/ledger_linux.go create mode 100644 core/services/addons/runtime/ledger_other.go create mode 100644 core/services/addons/runtime/supervisor.go create mode 100644 core/services/addons/runtime/supervisor_test.go create mode 100644 core/services/addons/runtime_integration_test.go create mode 100644 core/services/addons/state.go create mode 100644 core/services/addons/storage.go create mode 100644 core/services/addons/storage_test.go create mode 100644 core/services/addons/toolargs/toolargs.go create mode 100644 core/services/addons/toolargs/toolargs_suite_test.go create mode 100644 core/services/addons/toolargs/toolargs_test.go create mode 100644 core/services/addons/tools/adapter.go create mode 100644 core/services/addons/tools/adapter_test.go create mode 100644 core/services/addons/tools/consumer_matrix_test.go create mode 100644 core/services/addons/tools/consumers.go create mode 100644 core/services/addons/tools/definitions_test.go create mode 100644 core/services/addons/tools/executor_test.go create mode 100644 core/services/addons/tools/tools_suite_test.go create mode 100644 core/services/addons/tools_test.go create mode 100644 core/services/addons/updates.go create mode 100644 core/services/addons/updates/checker.go create mode 100644 core/services/addons/updates/checker_test.go create mode 100644 core/services/addons/updates/semver.go create mode 100644 core/services/addons/updates/status.go create mode 100644 core/services/addons/updates_test.go create mode 100644 core/services/finetune/export_name_test.go create mode 100644 core/services/quantization/import_name_test.go create mode 100644 docs/content/development/addon-testing.md create mode 100644 docs/content/features/addons.md create mode 100644 docs/content/features/tables.md create mode 100644 docs/superpowers/plans/2026-10-16-addons-navigation.md create mode 100644 docs/superpowers/specs/2026-10-16-addons-navigation-design.md create mode 100644 gallery/addons/README.md create mode 100644 gallery/addons/face.yaml.tmpl create mode 100644 gallery/addons/fine-tuning.yaml.tmpl create mode 100644 gallery/addons/index.yaml create mode 100644 gallery/addons/quantization.yaml.tmpl create mode 100644 gallery/addons/studio.yaml.tmpl create mode 100644 gallery/addons/tables.yaml.tmpl create mode 100644 gallery/addons/talk.yaml.tmpl create mode 100644 gallery/addons/voice.yaml.tmpl create mode 100644 pkg/addons/contract/contract_suite_test.go create mode 100644 pkg/addons/contract/contributions.go create mode 100644 pkg/addons/contract/hostops.go create mode 100644 pkg/addons/contract/hostops_test.go create mode 100644 pkg/addons/contract/limits.go create mode 100644 pkg/addons/contract/manifest.go create mode 100644 pkg/addons/contract/manifest_test.go create mode 100644 pkg/addons/contract/protocol.go create mode 100644 pkg/addons/contract/protocol_test.go create mode 100644 pkg/addons/contract/tooldefs.go create mode 100644 pkg/addons/contract/tooldefs_test.go create mode 100644 pkg/addons/model/proposalflow/README.md create mode 100644 pkg/addons/model/proposalflow/cmd/viewer/main.go create mode 100644 pkg/addons/model/proposalflow/cmd/viewer/viewer.tmpl.html create mode 100644 pkg/addons/model/proposalflow/explore.go create mode 100644 pkg/addons/model/proposalflow/explore_test.go create mode 100644 pkg/addons/model/proposalflow/gen.go create mode 100644 pkg/addons/model/proposalflow/reduce.go create mode 100644 pkg/addons/model/proposalflow/reduce_test.go create mode 100644 pkg/addons/model/proposalflow/suite_test.go create mode 100644 pkg/addons/model/proposalflow/traces.go create mode 100644 pkg/addons/model/proposalflow/traces_test.go create mode 100644 pkg/addons/model/proposalflow/viewer.html create mode 100644 pkg/addons/packaging/build.go create mode 100644 pkg/addons/packaging/catalog.go create mode 100644 pkg/addons/packaging/channel.go create mode 100644 pkg/addons/packaging/config.go create mode 100644 pkg/addons/packaging/helpers_test.go create mode 100644 pkg/addons/packaging/layout.go create mode 100644 pkg/addons/packaging/merge.go create mode 100644 pkg/addons/packaging/packaging_test.go create mode 100644 pkg/addons/packaging/platforms_test.go create mode 100644 pkg/addons/packaging/policy.go create mode 100644 pkg/addons/packaging/registry.go create mode 100644 pkg/addons/packaging/run.go create mode 100644 pkg/addons/packaging/verify.go create mode 100644 pkg/oci/cosignverify/offline_internal_test.go create mode 100644 pkg/utils/model_name.go create mode 100644 pkg/utils/model_name_test.go create mode 100755 scripts/addons/channel.sh create mode 100644 scripts/addons/cmd/addonpkg/main.go create mode 100644 scripts/addons/esm_ui.sh create mode 100755 scripts/addons/foundation.sh create mode 100755 scripts/addons/foundation_gate.sh create mode 100644 scripts/addons/gallery_ui.sh create mode 100644 scripts/addons/host_slots.sh create mode 100644 scripts/addons/leaks.sh create mode 100755 scripts/addons/package.sh create mode 100755 scripts/addons/package_test.sh create mode 100644 scripts/addons/prepare.sh create mode 100644 scripts/addons/prerequisites_test.sh create mode 100755 scripts/addons/recovery.sh create mode 100644 scripts/addons/suites.json create mode 100755 scripts/addons/tables_acceptance.sh create mode 100755 scripts/addons/tables_csv.sh create mode 100755 scripts/addons/tables_gate.sh create mode 100644 scripts/addons/tables_tools.sh create mode 100644 scripts/addons/tables_ui.sh create mode 100644 scripts/addons/ui_cases.py create mode 100755 scripts/addons/updates.sh create mode 100644 scripts/addons/verify.sh create mode 100644 tests/addons/fixture/cmd/main.go create mode 100644 tests/addons/fixture/ui/bare/manifest.json create mode 100644 tests/addons/fixture/ui/bare/ui.js create mode 100644 tests/addons/fixture/ui/cssb/manifest.json create mode 100644 tests/addons/fixture/ui/cssb/style.css create mode 100644 tests/addons/fixture/ui/cssb/ui.js create mode 100644 tests/addons/fixture/ui/slots/manifest.json create mode 100644 tests/addons/fixture/ui/slots/ui.js create mode 100644 tests/addons/fixture/ui/slots/ui/style.css create mode 100644 tests/addons/fixture/ui/uifixture-2/manifest.json create mode 100644 tests/addons/fixture/ui/uifixture-2/ui.js create mode 100644 tests/addons/fixture/ui/uifixture-2/ui/style.css create mode 100644 tests/addons/fixture/ui/uifixture/manifest.json create mode 100644 tests/addons/fixture/ui/uifixture/ui.js create mode 100644 tests/addons/fixture/ui/uifixture/ui/style.css create mode 100644 tests/addons/fixtures/contracts/app.json create mode 100644 tests/addons/fixtures/contracts/chat.json create mode 100644 tests/addons/fixtures/contracts/envelope.json create mode 100644 tests/addons/fixtures/contracts/headless.json create mode 100644 tests/addons/fixtures/contracts/incompatible.json create mode 100644 tests/addons/fixtures/contracts/page.json create mode 100644 tests/addons/fixtures/contracts/service.json create mode 100644 tests/addons/fixtures/packages/manifest.json create mode 100644 tests/addons/fixtures/signing/signing.go create mode 100644 tests/addons/foundation/cluster/catalog.go create mode 100644 tests/addons/foundation/cluster/cluster.go create mode 100644 tests/addons/foundation/cluster/config.go create mode 100644 tests/addons/foundation/cluster/nodes.go create mode 100644 tests/addons/foundation/cluster/packages.go create mode 100644 tests/addons/foundation/cluster/server.go create mode 100644 tests/addons/foundation/cluster/teardown_test.go create mode 100644 tests/addons/foundation/cmd/foundation-cluster/main.go create mode 100644 tests/addons/foundation/foundation_suite_test.go create mode 100644 tests/addons/foundation/foundation_test.go create mode 100644 tests/addons/foundation/recovery_failure_test.go create mode 100644 tests/addons/foundation/recovery_test.go create mode 100644 tests/addons/foundation/service_test.go create mode 100644 tests/addons/foundation/two_frontends_test.go create mode 100644 tests/addons/hostslots/hostslots_suite_test.go create mode 100644 tests/addons/hostslots/hostslots_test.go create mode 100644 tests/addons/packaging/packaging_test.go create mode 100644 tests/addons/recovery/host_test.go create mode 100644 tests/addons/recovery/recovery_test.go create mode 100644 tests/addons/tables/application_test.go create mode 100644 tests/addons/tables/databases_test.go create mode 100644 tests/addons/tables/rows_test.go create mode 100644 tests/addons/tables/schema_test.go create mode 100644 tests/addons/tables/tables_suite_test.go create mode 100644 tests/addons/tables/tools_test.go create mode 100644 tests/addons/tablesacceptance/.gitignore create mode 100644 tests/addons/tablesacceptance/authorization_test.go create mode 100644 tests/addons/tablesacceptance/cmd/tables-cluster/main.go create mode 100644 tests/addons/tablesacceptance/concurrency_test.go create mode 100644 tests/addons/tablesacceptance/evidence_test.go create mode 100644 tests/addons/tablesacceptance/faults_test.go create mode 100644 tests/addons/tablesacceptance/helpers_test.go create mode 100644 tests/addons/tablesacceptance/recovery_test.go create mode 100644 tests/addons/tablesacceptance/refusals_test.go create mode 100644 tests/addons/tablesacceptance/replicas_test.go create mode 100644 tests/addons/tablesacceptance/tables_suite_test.go create mode 100644 tests/addons/tablesacceptance/tablespkg/tablespkg.go create mode 100644 tests/addons/tablesadmin/apply_test.go create mode 100644 tests/addons/tablesadmin/delete_test.go create mode 100644 tests/addons/tablesadmin/docs_test.go create mode 100644 tests/addons/tablesadmin/export_test.go create mode 100644 tests/addons/tablesadmin/host_test.go create mode 100644 tests/addons/tablesadmin/parity_test.go create mode 100644 tests/addons/tablesadmin/resources_test.go create mode 100644 tests/addons/tablesadmin/selection_test.go create mode 100644 tests/addons/tablesadmin/store_test.go create mode 100644 tests/addons/tablesexport/export_suite_test.go create mode 100644 tests/addons/tablesexport/export_test.go create mode 100644 tests/e2e-ui/addons_fixture.go diff --git a/.agents/backend-signing.md b/.agents/backend-signing.md index 6d388dbba..f52be93f0 100644 --- a/.agents/backend-signing.md +++ b/.agents/backend-signing.md @@ -150,3 +150,10 @@ compromised and we've shipped malicious signed images: page if/when added. - **Tarball/HTTP backend signing.** Cosign can sign arbitrary blobs, but for now non-OCI backends keep using the `sha256:` field in YAML. + +Embedded consumers with host-provisioned private Sigstore trust can use +`cosignverify.NewVerifierWithTrustedMaterial`. It bypasses public TUF fetching, +not signature, identity, SCT, or Rekor checks. Never construct this input from a +registry attachment or catalog response. `NewVerifier` retains its public TUF +default. Hermetic cryptographic regressions live in `offline_internal_test.go`; +the optional live registry tests remain separate and are not acceptance evidence. diff --git a/.agents/localai-assistant-mcp.md b/.agents/localai-assistant-mcp.md index bd391164c..deeb02837 100644 --- a/.agents/localai-assistant-mcp.md +++ b/.agents/localai-assistant-mcp.md @@ -92,6 +92,32 @@ The in-process MCP server runs inside the same LocalAI binary that serves chat. The user chose KISS. Every mutating tool has a safety rule (`prompts/10_safety.md` rule 1) that requires the LLM to summarise the action and wait for explicit user confirmation before calling it. There is no `plan_*`/`apply_*` two-step in code. If you add a mutating tool, do **not** add per-tool confirmation logic in Go — instead, list the new tool name in `prompts/10_safety.md` so the LLM knows it falls under the confirmation rule. +## Scoped exception: add-on tools are not assistant tools + +The prompt-only confirmation above applies to the admin catalog in +`pkg/mcp/localaitools/` and to nothing else. Tools of installed add-ons (for +example Tables) follow a different, code-enforced rule, documented in +[docs/content/features/tables.md](../docs/content/features/tables.md): + +- They are never registered in this catalog and no `/api/addons` route appears + in `toolToHTTPRoute`. `TestToolHTTPRouteMappingComplete` stays unchanged, and + the consumer matrix test (`core/services/addons/tools`) fails if a tool name + or route of the add-on surface appears here. +- A change proposed by an add-on tool is a stored proposal. Only the proposer's + own browser session confirms it, with the digest of what the host card showed. + A model's "yes" or an argument named `approved` is never authority. +- REST-to-MCP parity is intentionally partial for add-ons: the lifecycle + mutations (install, enable, disable, upgrade, uninstall, library changes) are + deliberately **not** exposed as assistant tools, because the assistant is one + process-wide server with no per-call principal and would inherit prompt-only + authorization. Do not add them without a per-call actor and a host approval + card. +- New code that discovers or executes tools must be classified in + `tools.Consumers` (`core/services/addons/tools/consumers.go`) as route or + reject. The matrix test scans the sources and fails for an unclassified file. + +Do not rewrite the unrelated assistant tools to match this rule. + ## Distributed mode The in-memory MCP server runs only on the head node (where the chat handler runs). `inproc.Client` wraps services that are already distributed-aware (`GalleryService` coordinates with workers; `ListNodes` reads the NATS-populated registry). No NATS routing of MCP tools — the admin surface lives on the head, period. diff --git a/.github/workflows/addons-package.yml b/.github/workflows/addons-package.yml new file mode 100644 index 000000000..043bb97b0 --- /dev/null +++ b/.github/workflows/addons-package.yml @@ -0,0 +1,426 @@ +--- +name: 'add-on packages' + +# Builds, checks, signs and publishes the first-party add-on OCI packages +# (Tables, Voice, Face, Fine-tuning and Quantization). This workflow is +# independent of the backend image workflows: it has its own registry input, +# its own signer identity and its own jobs. +# +# Event policy (the rules live in scripts/addons/channel.sh): +# pull_request validate and build legs, publish nothing, no secrets +# push to master publish the development channel +# push to other branch publish nothing +# push of a v tag prepare the immutable version, move no alias +# release published promote latest, for a stable non-draft release only +# +# Every add-on has its own platform file (addons//package/platforms.yaml), +# its own package legs and its own repository. Tables publishes to +# ADDON_OCI_REPOSITORY, which is the repository /tables. Each other +# add-on publishes to /, where the namespace is +# ADDON_OCI_REPOSITORY without its last path element. A publishing event fails +# for an add-on until one of its platforms is enabled. +# +# Aliases move only in the promote job, after every platform leg passed, the +# index was pushed by digest, signed and verified. + +on: + push: + branches: ['**'] + tags: ['v*'] + paths: + - 'addons/**' + - 'pkg/addons/**' + - 'scripts/addons/**' + - 'gallery/addons/**' + - '.github/workflows/addons-package.yml' + pull_request: + paths: + - 'addons/**' + - 'pkg/addons/**' + - 'scripts/addons/**' + - 'gallery/addons/**' + - '.github/workflows/addons-package.yml' + release: + types: [published] + +permissions: + contents: read + +jobs: + validate: + name: validate package rules + runs-on: ubuntu-24.04 + timeout-minutes: 20 + permissions: + contents: read + strategy: + fail-fast: true + matrix: + addon: [tables, voice, face, fine-tuning, quantization, studio, talk] + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + - uses: actions/setup-node@v6 + with: + node-version: '22' + - name: Install the host UI build tools + # The package checks build the add-on UI with the bundler the host UI + # already installs. + run: npm ci --prefix core/http/react-ui + - name: Generate the gRPC stubs the packaging checks compile + # The generated stubs are not in the tree and the Go packages of the + # host import them. Same pinned tools as the test workflow. + if: matrix.addon == 'tables' + run: | + curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip + sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc + rm protoc.zip + go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2 + go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af + PATH="$PATH:$HOME/go/bin" make protogen-go + - name: Build the UI of this add-on + env: + ADDON: ${{ matrix.addon }} + run: node "addons/$ADDON/ui/build.mjs" + - name: Fill the module cache the packaging checks read + # package_test.sh runs offline (GOPROXY=off) against its own cache. + # The suite covers every add-on, so one leg runs it. + if: matrix.addon == 'tables' + run: | + GOPATH="$PWD/.cache/addons/go" GOMODCACHE="$PWD/.cache/addons/go/pkg/mod" \ + GOCACHE="$PWD/.cache/addons/go-build" go mod download + - name: Run the packaging checks + if: matrix.addon == 'tables' + run: sh scripts/addons/package_test.sh + + plan: + name: plan the event + runs-on: ubuntu-24.04 + timeout-minutes: 10 + permissions: + contents: read + env: + ADDON_OCI_REPOSITORY: ${{ vars.ADDON_OCI_REPOSITORY }} + outputs: + publish: ${{ steps.plan.outputs.publish }} + immutable: ${{ steps.plan.outputs.immutable }} + aliases: ${{ steps.plan.outputs.aliases }} + channel: ${{ steps.plan.outputs.channel }} + version: ${{ steps.plan.outputs.version }} + matrix: ${{ steps.plan.outputs.matrix }} + addons: ${{ steps.plan.outputs.addons }} + has_platforms: ${{ steps.plan.outputs.has_platforms }} + registry: ${{ steps.plan.outputs.registry }} + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + - name: Decide what this event may publish + id: plan + env: + EVENT: ${{ github.event_name }} + REF: ${{ github.ref }} + SHA: ${{ github.sha }} + DRAFT: ${{ github.event.release.draft || false }} + PRERELEASE: ${{ github.event.release.prerelease || false }} + run: | + set -eu + flags="" + [ "$DRAFT" = "true" ] && flags="$flags --draft" + [ "$PRERELEASE" = "true" ] && flags="$flags --prerelease" + # shellcheck disable=SC2086 + sh scripts/addons/channel.sh --event "$EVENT" --ref "$REF" --sha "$SHA" $flags > decision.txt + cat decision.txt + version=$(sed -n 's/^version=//p' decision.txt) + publish=$(sed -n 's/^publish=//p' decision.txt) + [ -n "$version" ] || version="0.0.0-dev.${SHA%"${SHA#????????????}"}" + if [ "$publish" = "true" ]; then + # Fail closed: a publishing event needs the namespace input and a + # non-empty release platform matrix. There is no default registry. + if [ -z "$ADDON_OCI_REPOSITORY" ]; then + echo "ADDON_OCI_REPOSITORY is not configured; refusing to publish" >&2 + exit 1 + fi + fi + # One set of legs for each add-on, from its own platform file. Only + # the "no enabled platform" answer is tolerated, and only on an event + # that publishes nothing: that add-on gets no legs. On a publishing + # event it fails the plan. A missing or malformed platform file, or + # any other error, fails the plan on every event. + include='[]' + for id in tables voice face fine-tuning quantization studio talk; do + cfg="addons/$id/package/platforms.yaml" + err="matrix-$id.err" + if rows=$(go run ./scripts/addons/cmd/addonpkg matrix --config "$cfg" 2>"$err"); then + : + elif grep -q 'no enabled platform' "$err"; then + if [ "$publish" = "true" ]; then + echo "add-on $id has no enabled platform; refusing to publish" >&2 + exit 1 + fi + continue + else + echo "cannot read the platform matrix of add-on $id" >&2 + cat "$err" >&2 + exit 1 + fi + include=$(printf '%s' "$rows" | jq -c --argjson all "$include" --arg addon "$id" '$all + [.include[] | . + {addon: $addon}]') + done + matrix=$(printf '%s' "$include" | jq -c '{include: .}') + addons=$(printf '%s' "$include" | jq -c 'map(.addon) | unique') + has=$(printf '%s' "$include" | jq 'length > 0') + { + cat decision.txt + echo "version=$version" + echo "matrix=$matrix" + echo "addons=$addons" + echo "has_platforms=$has" + echo "registry=${ADDON_OCI_REPOSITORY%%/*}" + } >> "$GITHUB_OUTPUT" + + package: + name: package ${{ matrix.addon }} ${{ matrix.platform }} + needs: [plan, validate] + if: needs.plan.outputs.has_platforms == 'true' + runs-on: ${{ matrix.runner }} + timeout-minutes: 60 + permissions: + contents: read + strategy: + fail-fast: true + matrix: ${{ fromJSON(needs.plan.outputs.matrix) }} + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + - uses: actions/setup-node@v6 + with: + node-version: '22' + - name: Build the UI + env: + ADDON: ${{ matrix.addon }} + run: | + npm ci --prefix core/http/react-ui + node "addons/$ADDON/ui/build.mjs" + - name: Package this platform + env: + ADDON: ${{ matrix.addon }} + VERSION: ${{ needs.plan.outputs.version }} + PLATFORM: ${{ matrix.platform }} + SHA: ${{ github.sha }} + run: | + sh scripts/addons/package.sh --config "addons/$ADDON/package/platforms.yaml" --layout "layout" --version "$VERSION" --platform "$PLATFORM" --revision "$SHA" + - name: Native runtime check + env: + ADDON: ${{ matrix.addon }} + VERSION: ${{ needs.plan.outputs.version }} + PLATFORM: ${{ matrix.platform }} + run: | + go run ./scripts/addons/cmd/addonpkg native-check --config "addons/$ADDON/package/platforms.yaml" --layout layout --platform "$PLATFORM" --version "$VERSION" + - uses: actions/upload-artifact@v7 + with: + name: layout-${{ matrix.addon }}-${{ matrix.slug }} + path: layout + if-no-files-found: error + retention-days: 7 + + publish: + name: publish and sign ${{ matrix.addon }} + needs: [plan, validate, package] + if: needs.plan.outputs.publish == 'true' + runs-on: ubuntu-24.04 + timeout-minutes: 60 + environment: addons-release + concurrency: + group: addons-publish-${{ matrix.addon }}-${{ needs.plan.outputs.immutable }} + cancel-in-progress: false + permissions: + contents: read + id-token: write + strategy: + # A leg that is pushing must finish, so one failure never cancels another. + fail-fast: false + matrix: + addon: ${{ fromJSON(needs.plan.outputs.addons) }} + env: + ADDON_OCI_BASE: ${{ vars.ADDON_OCI_REPOSITORY }} + ADDON: ${{ matrix.addon }} + # cosign gates --registry-referrers-mode=oci-1-1 behind this flag. + COSIGN_EXPERIMENTAL: '1' + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + - name: Choose the repository of this add-on + # Tables publishes to the configured repository. Every other add-on + # publishes to /. The later steps read + # ADDON_OCI_REPOSITORY. + run: | + set -eu + case "$ADDON_OCI_BASE" in + */*) ;; + *) echo "ADDON_OCI_REPOSITORY needs a namespace and a name" >&2; exit 1 ;; + esac + if [ "$ADDON" = tables ]; then + repo="$ADDON_OCI_BASE" + else + repo="${ADDON_OCI_BASE%/*}/$ADDON" + fi + echo "ADDON_OCI_REPOSITORY=$repo" >> "$GITHUB_ENV" + - uses: actions/download-artifact@v8 + with: + pattern: layout-${{ matrix.addon }}-* + path: legs + - name: Join the platform legs into one index + id: merge + env: + VERSION: ${{ needs.plan.outputs.version }} + SHA: ${{ github.sha }} + run: | + set -eu + go run ./scripts/addons/cmd/addonpkg merge --config "addons/$ADDON/package/platforms.yaml" \ + --out merged --version "$VERSION" --revision "$SHA" legs/* + - name: Registry login + uses: docker/login-action@v3 + with: + registry: ${{ needs.plan.outputs.registry }} + username: ${{ secrets.ADDON_OCI_USERNAME }} + password: ${{ secrets.ADDON_OCI_PASSWORD }} + - name: Push the platform digests and the index + id: push + env: + IMMUTABLE: ${{ needs.plan.outputs.immutable }} + run: | + set -eu + go run ./scripts/addons/cmd/addonpkg publish --config "addons/$ADDON/package/platforms.yaml" \ + --layout merged --repository "$ADDON_OCI_REPOSITORY" --tag "$IMMUTABLE" --adopt-existing | tee push.txt + echo "digest=$(sed -n 's/^digest=//p' push.txt)" >> "$GITHUB_OUTPUT" + echo "children=$(sed -n 's/^children=//p' push.txt)" >> "$GITHUB_OUTPUT" + - uses: sigstore/cosign-installer@v3 + with: + cosign-release: 'v2.6.5' + - name: Sign by digest + id: sign + env: + DIGEST: ${{ steps.push.outputs.digest }} + run: | + set -eu + cosign sign --yes --recursive \ + --new-bundle-format \ + --registry-referrers-mode=oci-1-1 \ + "${ADDON_OCI_REPOSITORY}@${DIGEST}" + - name: Verify the signature with the exact add-on workflow identity + id: verify + env: + DIGEST: ${{ steps.push.outputs.digest }} + CHILDREN: ${{ steps.push.outputs.children }} + IDENTITY: https://github.com/${{ github.repository }}/.github/workflows/addons-package.yml@${{ github.ref }} + ISSUER: https://token.actions.githubusercontent.com + run: | + set -eu + for d in "$DIGEST" $(echo "$CHILDREN" | tr ',' ' '); do + cosign verify --new-bundle-format \ + --certificate-identity "$IDENTITY" \ + --certificate-oidc-issuer "$ISSUER" \ + "${ADDON_OCI_REPOSITORY}@${d}" > /dev/null + done + - name: Keep the digest for the promote job + # A matrix job cannot hand one output to the next job, so the digest of + # each add-on travels in an artifact of its own. + env: + DIGEST: ${{ steps.push.outputs.digest }} + run: printf '%s\n' "$DIGEST" > digest.txt + - uses: actions/upload-artifact@v7 + with: + name: digest-${{ matrix.addon }} + path: digest.txt + if-no-files-found: error + retention-days: 7 + + promote: + name: promote channel aliases ${{ matrix.addon }} + needs: [plan, validate, publish] + if: needs.plan.outputs.publish == 'true' && needs.plan.outputs.aliases != '' + runs-on: ubuntu-24.04 + timeout-minutes: 20 + environment: addons-release + # One promotion at a time per channel, and a queued run never cancels a + # running one. The freshness check below makes a slow, older run a no-op. + concurrency: + group: addons-promote-${{ matrix.addon }}-${{ needs.plan.outputs.channel }} + cancel-in-progress: false + permissions: + contents: read + strategy: + fail-fast: false + matrix: + addon: ${{ fromJSON(needs.plan.outputs.addons) }} + env: + ADDON_OCI_BASE: ${{ vars.ADDON_OCI_REPOSITORY }} + ADDON: ${{ matrix.addon }} + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + persist-credentials: false + - uses: actions/setup-go@v6 + with: + go-version-file: go.mod + - name: Choose the repository of this add-on + # The same rule as in publish: Tables keeps the configured repository, + # every other add-on uses /. + run: | + set -eu + case "$ADDON_OCI_BASE" in + */*) ;; + *) echo "ADDON_OCI_REPOSITORY needs a namespace and a name" >&2; exit 1 ;; + esac + if [ "$ADDON" = tables ]; then + repo="$ADDON_OCI_BASE" + else + repo="${ADDON_OCI_BASE%/*}/$ADDON" + fi + echo "ADDON_OCI_REPOSITORY=$repo" >> "$GITHUB_ENV" + - uses: actions/download-artifact@v8 + with: + name: digest-${{ matrix.addon }} + path: digest + - name: Registry login + uses: docker/login-action@v3 + with: + registry: ${{ needs.plan.outputs.registry }} + username: ${{ secrets.ADDON_OCI_USERNAME }} + password: ${{ secrets.ADDON_OCI_PASSWORD }} + - name: Move the aliases + env: + ALIASES: ${{ needs.plan.outputs.aliases }} + VERSION: ${{ needs.plan.outputs.version }} + SHA: ${{ github.sha }} + REF: ${{ github.ref }} + EVENT: ${{ github.event_name }} + run: | + set -eu + DIGEST=$(tr -d '[:space:]' < digest/digest.txt) + case "$DIGEST" in + sha256:????????????????????????????????????????????????????????????????) ;; + *) echo "the digest of $ADDON is missing or malformed" >&2; exit 1 ;; + esac + for alias in $(echo "$ALIASES" | tr ',' ' '); do + go run ./scripts/addons/cmd/addonpkg promote \ + --repository "$ADDON_OCI_REPOSITORY" --alias "$alias" \ + --digest "$DIGEST" --version "$VERSION" --revision "$SHA" \ + --ref "$REF" --event "$EVENT" --git-dir .git + done diff --git a/.gitignore b/.gitignore index 3572eaaf8..48ebf2eee 100644 --- a/.gitignore +++ b/.gitignore @@ -140,3 +140,4 @@ formal-verification/out/ /backend/go/gemxcpp/build-*/ /backend/go/gemxcpp/package/ /backend/go/gemxcpp/gemxcpp +addons/internal/uibuild/test/*/dist diff --git a/addons/face/cmd/face/main.go b/addons/face/cmd/face/main.go new file mode 100644 index 000000000..5531849e5 --- /dev/null +++ b/addons/face/cmd/face/main.go @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT + +// Command face is the executable of the Face add-on. It serves no tools: +// the add-on works in the browser, through the SDK and the core API. +package main + +import ( + "os" + + "github.com/mudler/LocalAI/addons/internal/idle" +) + +func main() { + os.Exit(idle.Run(os.Args)) +} diff --git a/addons/face/manifest.json b/addons/face/manifest.json new file mode 100644 index 000000000..9aba07e9c --- /dev/null +++ b/addons/face/manifest.json @@ -0,0 +1,25 @@ +{ + "schemaVersion": 1, + "id": "face", + "version": "0.1.0", + "executable": "face", + "os": "linux", + "arch": "amd64", + "host": { "min": 1, "max": 1 }, + "api": { "min": 1, "max": 1 }, + "protocol": { "min": 1, "max": 1 }, + "dataFormat": { "min": 1, "max": 1 }, + "capabilities": { + "ui": { + "entry": "ui/face.js", + "css": ["ui/face.css"], + "sdk": { "min": 2, "max": 2 }, + "pages": ["recognition"], + "navigation": [], + "mainPage": "recognition", + "icon": "ui/icon.png", + "title": "Face", + "description": "Find out whose face is in a photo, or whether two photos show the same person." + } + } +} diff --git a/addons/face/package/platforms.yaml b/addons/face/package/platforms.yaml new file mode 100644 index 000000000..23666df9d --- /dev/null +++ b/addons/face/package/platforms.yaml @@ -0,0 +1,38 @@ +# Release platform matrix of the face add-on package. +# +# A platform ships only while "enabled" is true. Enable a platform in the same +# change that shows its native check passing on a runner of that platform; +# until then it stays listed and disabled. The release workflow fails closed +# when no platform is enabled, and it never builds a platform that is not +# enabled here. +# +# runner: the GitHub runner label that builds and checks the platform +# natively. Required for an enabled platform. +# native_check: "executable-start": the packaged executable is extracted on +# the native runner and started. Required for an enabled platform. +# runtime: files besides the executable that the platform needs at run +# time, relative to the package root. They are taken from the +# runtime directory of the build and must exist. +schema: 1 +addon: face +executable: face +manifest: addons/face/manifest.json +# The directory whose content is laid at the package root (ui/face.js ...). +ui: addons/face/ui/dist +platforms: + - platform: linux/amd64 + enabled: true + runner: ubuntu-24.04 + native_check: executable-start + - platform: linux/arm64 + enabled: true + runner: ubuntu-24.04-arm + native_check: executable-start + - platform: darwin/arm64 + enabled: true + runner: macos-14 + native_check: executable-start + - platform: darwin/amd64 + enabled: true + runner: macos-15-intel + native_check: executable-start diff --git a/addons/face/ui/.gitignore b/addons/face/ui/.gitignore new file mode 100644 index 000000000..1225aa2fa --- /dev/null +++ b/addons/face/ui/.gitignore @@ -0,0 +1,2 @@ +# Written by build.mjs; the package build makes it. +/dist/ diff --git a/addons/face/ui/build.mjs b/addons/face/ui/build.mjs new file mode 100644 index 000000000..99db94154 --- /dev/null +++ b/addons/face/ui/build.mjs @@ -0,0 +1,13 @@ +import { fileURLToPath } from 'node:url' +import { dirname } from 'node:path' +import { buildAddonUI } from '../../internal/uibuild/uibuild.mjs' + +const here = dirname(fileURLToPath(import.meta.url)) + +export async function build() { + return buildAddonUI({ root: here, name: 'face', rgb: [20, 100, 90] }) +} + +if (process.argv[1] === fileURLToPath(import.meta.url)) { + build().then((dir) => console.log(`built ${dir}`), (e) => { console.error(e); process.exit(1) }) +} diff --git a/addons/face/ui/icon.png b/addons/face/ui/icon.png new file mode 100644 index 0000000000000000000000000000000000000000..a72b97f95f3766f71fb798cf8b37a70bfe462cf0 GIT binary patch literal 6999 zcmbVx^osqspB&4_#HQR9)~ZouA6b0Gg#bK;e2;9jYp z`a{RkiSr10g?b38{hd%SKZ+RNgBeDSJmiltAlAC0peGn3c<3Ua6PN_T-w1DIfxoUr zeCLv8eSOw$nX~gj?arja`~iN+^STG(^`s+O*>!P;p*6|#aUd8$pHFz9jv_oWj1Su#B^nAn1{aOV$ zX3QDYt`jgR_{)4j-4~3>gTiS-S)|;QEPh{G>eS?hrrH4y*q1Qn}Ep$P~-}ydEfrPyzwnQm>@&cJ-kPUg&+L5Trk=e69=`S6m!N>dD$q z`L$K%R=oQ_sf^A<3d^fkhkqMLGC5Qrz@QH4Sb|4Kp+BK*R zidBoysn47-RmG(&GI*{^lD7xSsoED|IemUxcUVby@uPX4KbJtSZ)j+AXao`=rbgC0 z19GSnBF})27L@g*F(u(LLB`in?BCXX+i6`mxfGGi zynsr)Ox`xX{*75U?IbqxhI1M*&BQ>RBaj;$^_Pw5PIuEex_8n=h+S^BVBs)nCkVi%>J) z+i_|nLr7UM+0%inVjG`xBQfQ_NG#p6#QitbUC2(0>4MHegWvPk=@^_xI#%S*z}&#s z0WysuM1FUHk@S2vVwOz=(osWiYwNB;xA&MzPzSIsF1#UdVF^QWq(CnES`k7U!YM&~MW6GAsX}QJtn&r8sd|)`N zUq*qY4AyO0^q_$E;8-wT2oWp@PJ6=4hCs@h5%AyttmwnNb-yJe;0dS7I4NB)vK?Xf zM`SU=-1joF2Y|JE(<)F4TgwI&A@|J7f({cvc=;?y4u1??z`TIGuld!^7GA5Vf;LRc z&Mk7Q|5O*0@KK^J)BkJ_U?q{3bKem>%`2%{Xy62_hiV(KE({cSmT(R&ZUTbDp*_^gb=uY;T?e3LiquG#%qpu9Bss}XMhMEf;fUa|j(qUNXj zfMifg#VV)EFv1d-3eilDrO{xSWf^dc81EGA!WbA&f!bL?NEB+v42Jcw5Pp5|$`fgx(w5ZB?Moh64&@-2k&IKNv{-Ab|6X0m&>|Kmk@F zD*z@6$46k={vTW_l@pdaQ#d~yj(q1ZXB)p3dmutZ>yv@`nF4XW2bO_zF`1Hi{c#na z9(xbY1k@;|o>37}`whAo6l{yna zn3OO`?$6*|hsBW@qKvWYq~X)mwAB_5w9kI;=EgJsQzgyy*Fb0?>R{{uvqPS*gY|^% zSD^NyPoKQQt?_Xt;nw~XD&tvNARe{vBmuE3*7&%+qCp~*Sf5}!ci0pe$tCr?@3cZT zFH8sl`%B3y;WnXpZ1;!IGe2a#s60Lc3+sK!mcbZ7O$P^9SoKbKZyz6sJ}vizY;W$e znGP28Zav;(P5w;C5-bRkm)QwL!Nt#cjA?HAQLkK(0w3m*hA!9=umXT%x#id;9J{bp z_^O;$E)3;L-#}K43yLQH!F9?vZJOtD}t@n`Lfi>SLEvV+oNXT@%g$ zqA7Cx)&jF_EOb+^m^3J>?^!I5^loR(XYrLD##M>izlB?bM&tHzGovk;s66|NzI~3pbqm1*I1Ml0afDh0FHKm$7jiXyx?TD)YXI7}a zI8bt0(G&bbp83jxJ$F!IcIW-M{aGZ_{;};)rr;r-);XWfSt>dpv2ky$^632j`DT$g z6X6sDzg1Ce!e#Hs;C5T>X#e;2g!5$gSw#Kbe9%Gog+a5X8-duk0|ZSDM#M_Y`4Iab zcy7Y|>+J1i=1-#Y@5@V$&p^7{Y2v*Ocf7tVSOAlqlC>vklDoRO4u7LGDqU~IQl3^n zL;52^b)g$uEr?lFQ)&7O^qaqB>xFhlUhED60Qb0J6=&mRtM8Y! zH?7X2fp`wfZvTyh4+8x-TfxkYxtqmxdBJB>ZkDo81DAzWbuyo(-*vwisX|7KY+v3L9s7YUvw-{)YGi;(m?26{LzW(P2 z*5m`O2naHb#teFV#{W0{aF|IGKpYlrYcz2h7}#P7XA!7n`ShMGH*B=VL~O<`oy+vz z3D-ohX;G|de_GtZ<+(+34T(->2P5tX9fbJucBXzb z@aX^6(ic8;oK{<7Gw>U3P8Sv@#F(_@jU-)m@>sdLjJhZc4JMj4nOW_ z3eN67b2Hcywl~}V zHb<1?>v+O2>NfIyZP|=L!i~UM5{yv0cDv{q{))mY2GY>E(lU|-|D4>hR#Oqoe*f+a z$Pi-i-Al-Qtn3wi(9ga2NA>79U+$BS9swUw*E{4~)#8$mz2SiH;Yjh@)f)5qQD~#U zhk4FKUC%L-pc8{l0gs~jo%wk+&=e8wKC9biw>9oWFGNS#a9JGVKrzqRfgrkmuSJr- z&!5{Ici)&F>Iu0fA;SY)7zxp_Sv{w@R!zcE9=p#K75@GT&l=8sA<3X2O;HJPqSJLk zEJ|j#E_j}y1aJJ`FaH_I5vCA27N+>RKpEMqGVUVqQoo4OxVQ98Np#Tl_YI7&f0G04 zIMZ}UG@);QWYO1p9DYR44c1>ZMVOy%jw*unW5QaZh&_y*LUNEsc%n|o9GFx6;JUYo^Q z(Kx)xKYr2dAFbB%6eZV|zb4C^c55MEmG-y3ZqTOZWp7}W^G1OFljWe~3I8};P9vOR z$^=+muXc;BYEBmzJMx?g0a<}F8oF3O`Rw5S9>Yhet8dL3u(hi^!Ijz2hBNlI63Td53 zSK67lZ#zF}DL;y*3g*t8zN|R*yCP})TS|nQu!|AVR;+Sq#VZA+R5oO~cL{n) zMkamfUK)CnCGWdSxNO!ZA9tOenPU~`-fwp4I%1>6#t9bqny;mJ6U|4(p84>Otns5l zj=cauQ1lr$Q7XeY@B%q4<)9EhpX<<}PRv%lI$!X>*k;C;NZs~JMt|VN3n3lO9=)gO zPzS>j(P_+0l*O*nd5ahM^OEVo#765R(T;==9OebV>RqigZxs417 z5ec(C;G=5$;z_7|h3{TC!%!jlvBv+&(o?f+Y`Cbka$6ffOevJ5-*j|UA3L5p3Y*F|?dNyvV3^^n z*ZS^=2d=&%G_7B}`2cS03sQ$SA=wu+fCD+Y0g+5=>mv zX8jepzWSY?5c^j>i&S0`F*94$A6kCf2rhIDDP4@^sbmWPc&gPG^R(xvDZgFvF|fMX z6AJOTJy{MNt84q?j(I6qb=hQ<;Ei24h>&c80Gf;b`r114R`*qO37&b8O?AsO8gTh_ z+psfz(X1(c|J$xX3@(ZP8APGLk;oqQ!_V66iYKmu zgCRNfOZ0HX^H&3-d*};{@V3-t#HbZY9H5$V&o_`bHeAMoU_i^}A%DHd zCZ0LZT+<)_wIR3GmG3?mKC62riYlAu2FBXhJN9nWW-9B_&d|T#aM{p8reA7*tkEfY z0DB81YRV~^KbftV7dfNa9}6hE%m~0+W&$IaD3{^4M@``!`Z;A-2CB3m=r?UYl0vtI zk~Lp4!8OK&Jvb2P{rybYt;@{3Fhn{4OY=_5gN&{GoS<2A?FPqEWty#k^Y!NONIRn@ z7JmVoS?;P$4e5+3k31iB|2#AN6QcwdI3T4t_Up^(SqF_h(q*%LDz~J_Ex$ngO0Mai zJr?hG53t{cX$Z73o3C(KY4-3A!sWQ3Zk`Nlv)RIzP4_zLZ|T0ogK|}zdcXSKcoc<+ z-PG7I6x!LlFJwJmTKOpkHMM`QTdfG%umtQhulBJuTT8=V(-wUtR~X%rp`6)bO7;OV z=Ht)czH6V;_EO>YaDZ+%HA@Ybz2(z=*EIT^suebGkXaIk)JdQ5A-OLJ0SC~W72sWD9-TdHb;Hj0savU}AlUzCFI8v2 ztx&$KZMfmWlzERIXv|Hk^Ud`LTBmDR#;>`;=KeB0{H-$lb(Ju5+PP2967bH`uey(W z=IP{9Ut}AF{FZ_)hJxPVOVZ8f zdH4rff|~zGFi{&+iG2IGrl;gJIPohI4s^i|rDwyWny0W%!(MCeS*vA{Mrz7Rwc%NK z^@b|XUIWyhowCx5W4&rfe3pBNX_4t5f+Za99KGiIN?Yjwfx!a{GSBDBPho+ zb(?}-tzc85+5_7nVRj3ET2}W~+|D_$D6O<$;kRVZ&gPFN<}VLpK$@YxsatkD=BCWz zg?c;YL?q>@SL3L)X=8z}edmqmg;e}N)NrZdgR-Rw#_5>G>wWXF3HzTHK9ldfOU9j0 zdJ}#;S)sU53Q(ZRn_)K1-eJj!dduC1H!kyZ1xBNe#U9tER(Uus!7-I5+ExD;KUcXF zlJHuhOlmSZi$EQr&v zz8VPfVkgsw0_4RGH{Z7xbb}tLivxe*Kz1d0EB>r>8usmFv_Vpk`;e#ufP*B7VXT#C z?`cq^cZDl;Zy7B(_>Nnwx#GA`=$a`H%&mf2yQyFUL2C3XjYfo`*%BFpBhTfk&%2;KvB z>7gyFettmWHPr&0&5I3e-m?8Qg|z1_=Zcb!sj+$nv@*)mV!(k7{_gWv+-$3T)X-4 zf!Tp!_#vGr8e(W`{tpV2Y)dxHZ^8CYg*@oM2%;R9Pti~L z?wzUPAlbU?p#R3B!)kPVyGnQv-!vY#!@&(Wpcd&F}^|8T&sq3XIVt`wh(dJaetH{D zBYL>>4S@ZVhO}+Wg+afZ(o|aWWLSwViUx9KwqG!2_-Y0IysNMbZ|5H`M!T3nm=Qv_ zQDCn#F5ig)X~9T_jve6vkpMw}cZG&KJr+*{$#LC_=FRUkIw_(BAURn+>VaN_NDL_E z?&Fp$SF?cABVWK7884$?xz~rjaRaDLVm>}jW`QVB*Ff>7D835Xn%rEj0S3p{-=;B-K=W2VhD$pjC{rtrQn zl4IMObEf^jU~E!EDGkg?R$=k=e~lD6(V289M%P-XbQL|Ezug61;IwXs<^6VCc>LdX zyZS{A1iPP3t;<_n;t;raYt9ntiG*q8eysAyQKl6=d2rs^poL&PrWdcvX^b{}Lh#}P z@xPu|? zrzUE6rst`M37~w`MOE92*eajlQ6V|&Z|m*%5P{M8xaPkELO_ZzRd!4U`Lp$3);U?lg?Q@B*8Z?0W~!hl$t$vDQazE0}Z6bIfOm`1kpz zBv6tf`Y&S`|MWdF;luF5IdVcYswg7cgHauP{cWRi9o~E%%RdG=EYk>DQ!irz3O_@3 zWnq8sUs!Jja}4y#1xoi2~3@`}{{vNC4AxChX+*~b)QBagi9qLTIk|8I1q;0^T z@EC%noMLG%tZe<*vf%X}SsR<iq5yARKV30MNhd8ZFmxy+ zU@a*UdJ>9B$+5WDUBa~hD$msw%H=FV{twwNK8FAR literal 0 HcmV?d00001 diff --git a/addons/face/ui/package.json b/addons/face/ui/package.json new file mode 100644 index 000000000..f3a51c012 --- /dev/null +++ b/addons/face/ui/package.json @@ -0,0 +1,8 @@ +{ + "name": "localai-face-ui", + "private": true, + "version": "0.1.0", + "description": "Native UI of the Face add-on: a standalone ES module that the host loads after its own build.", + "type": "module", + "scripts": { "build": "node build.mjs", "test": "node test/run.mjs" } +} diff --git a/addons/face/ui/src/FaceRecognition.jsx b/addons/face/ui/src/FaceRecognition.jsx new file mode 100644 index 000000000..bd5689cbc --- /dev/null +++ b/addons/face/ui/src/FaceRecognition.jsx @@ -0,0 +1,116 @@ +// The Faces page: who is this, same person, the people this browser knows and +// what is stored. Ported from pages/FaceRecognition.jsx. The known-faces list +// and the enrol sheet are the shared views inside the shared Workbench. +// Detect and analyze, and the raw faceprint, sit behind a disclosure. +import { createClipInput } from '../../../internal/identity-ui/src/ClipInput.jsx' +import { createWorkbench } from '../../../internal/identity-ui/src/Workbench.jsx' +import { createKnownList } from '../../../internal/identity-ui/src/KnownList.jsx' +import { createEnrolSheet } from '../../../internal/identity-ui/src/EnrolSheet.jsx' +import { createVerdict } from '../../../internal/identity-ui/src/Verdict.jsx' +import { createDistanceScale } from '../../../internal/identity-ui/src/DistanceScale.jsx' +import { createErrorNote } from '../../../internal/identity-ui/src/ErrorNote.jsx' +import { createModelNeeded } from '../../../internal/identity-ui/src/ModelNeeded.jsx' +import { createBoundingBoxCanvas } from '../../../internal/identity-ui/src/BoundingBoxCanvas.jsx' +import { createIdentityRegistry } from '../../../internal/identity-ui/src/registry.js' +import { createMoreFaceTools } from './MoreFaceTools.jsx' + +const CAP_FACE_RECOGNITION = 'FLAG_FACE_RECOGNITION' + +// `identity` is the shared helper module (identity.js), passed as in the Voice +// add-on. The Faces page has no tab counts, so it reads nothing from it. +// eslint-disable-next-line no-unused-vars +export function createFaceRecognition(ctx, api, identity) { + const { h, t, Icon, ui, viewer } = ctx + const { useCallback, useEffect, useMemo, useState } = ctx.React + + const { ClipInput } = createClipInput(ctx) + const { KnownList } = createKnownList(ctx) + const { Verdict } = createVerdict(ctx) + const { DistanceScale } = createDistanceScale(ctx) + const { ErrorNote } = createErrorNote(ctx) + const { useRegistry } = createIdentityRegistry(ctx) + const { BoundingBoxCanvas } = createBoundingBoxCanvas(ctx) + const { EnrolSheet } = createEnrolSheet(ctx, { ClipInput, ErrorNote, api }) + const { ModelNeeded } = createModelNeeded(ctx, api) + const { MoreFaceTools } = createMoreFaceTools(ctx, api) + const { Workbench } = createWorkbench(ctx, { ClipInput, KnownList, EnrolSheet, Verdict, DistanceScale, ErrorNote, useRegistry, BoundingBoxCanvas, api }) + + // Shown in place of a page whose feature the signed-in user may not use. It + // replaces a silent redirect: the person learns what is off and what turns it on. + function FeatureOff() { + return ( +
+
+ {t('off.badge')} +

{t('off.face.title')}

+

{t('off.face.text')}

+
+ {t('needed.whatTitle')} +

{t('off.face.what')}

+
+
+
+ ) + } + + // The face models that are installed: the capability list of the core, + // narrowed to the face-recognition flag, as the core hook useModels does. + function useFaceModels() { + const [state, setState] = useState({ models: [], loading: true }) + const load = useCallback(async (silent) => { + if (!silent) setState(s => ({ ...s, loading: true })) + try { + const data = await api.models.listCapabilities() + const models = (data?.data || []).filter(m => !m.disabled && m.capabilities?.includes(CAP_FACE_RECOGNITION)) + setState({ models, loading: false }) + } catch { + setState({ models: [], loading: false }) + } + }, []) + useEffect(() => { load(false) }, [load]) + return { ...state, refetch: useCallback(() => load(true), [load]) } + } + + function FacesPage({ path }) { + const isAdmin = viewer.isAdmin() + const [model, setModel] = useState(() => ctx.modelFrom(path) || '') + const { models, loading, refetch } = useFaceModels() + const names = useMemo(() => models.map(m => m.id), [models]) + const needModel = !loading && names.length === 0 + + return ( +
+ + + + ) : null} + /> + : null} + more={needModel ? null : ( +
+ {t('face.moreTitle')} {t('face.moreHint')} + +
+ )} + /> +
+ ) + } + + // The page is for a viewer who holds face_recognition (spec, "Classification": + // the gate of the page is the gate of the core route, and the library inside + // it has no stricter gate). The server checks every /v1/face/* call again. A + // viewer without the feature gets the notice and no call. + function FaceRecognition(props) { + if (!viewer.hasFeature('face_recognition')) return + return + } + + return { FaceRecognition } +} diff --git a/addons/face/ui/src/MoreFaceTools.jsx b/addons/face/ui/src/MoreFaceTools.jsx new file mode 100644 index 000000000..df70f8c6a --- /dev/null +++ b/addons/face/ui/src/MoreFaceTools.jsx @@ -0,0 +1,246 @@ +// The two developer tools behind the face workbench: detect faces and guess +// attributes (guesses, often wrong), and the raw faceprint. Ported from +// components/identity/MoreFaceTools.jsx; the shared widgets come from the SDK. +import { createBoundingBoxCanvas } from '../../../internal/identity-ui/src/BoundingBoxCanvas.jsx' + +const TABS = [ + { id: 'analyze', icon: 'chart-bar', label: 'Detect and analyze' }, + { id: 'embed', icon: 'code', label: 'Embedding' }, +] + +export function createMoreFaceTools(ctx, api) { + const { h, Fragment, Icon, ui } = ctx + const { useMemo, useState } = ctx.React + const { BoundingBoxCanvas } = createBoundingBoxCanvas(ctx) + + function MoreFaceTools({ model }) { + const [tab, setTab] = useState('analyze') + return ( +
+ +
+ {tab === 'analyze' && } + {tab === 'embed' && } +
+
+ ) + } + + function AnalyzeTab({ model }) { + const [img, setImg] = useState(null) + const [actions, setActions] = useState({ age: false, gender: false, emotion: false, race: false }) + const [antiSpoofing, setAntiSpoofing] = useState(false) + const [loading, setLoading] = useState(false) + const [error, setError] = useState(null) + const [result, setResult] = useState(null) + const [focusIdx, setFocusIdx] = useState(0) + + const submit = async (e) => { + e.preventDefault() + if (!model) { ctx.toast('Select a face model first', 'warning'); return } + if (!img) { ctx.toast('Add an image to analyze', 'warning'); return } + setLoading(true); setError(null); setResult(null); setFocusIdx(0) + try { + const data = await api.face.analyze({ + model, + img: img.dataUrl, + actions: Object.entries(actions).filter(([, v]) => v).map(([k]) => k), + anti_spoofing: antiSpoofing, + }) + setResult(data) + if (!data?.faces?.length) ctx.toast('No face detected in the image', 'warning') + } catch (err) { + setError(err.message) + } finally { + setLoading(false) + } + } + + const boxes = useMemo(() => (result?.faces || []).map((f, i) => ({ + x: f.region.x, y: f.region.y, w: f.region.w, h: f.region.h, + label: f.dominant_emotion || f.dominant_gender || `Face ${i + 1}`, + sublabel: f.age ? `~${Math.round(f.age)}y` : null, + tone: i === focusIdx ? 'accent' : 'default', + })), [result, focusIdx]) + + const faces = result?.faces || [] + const focus = faces[focusIdx] + + return ( +
+ + +
+ {loading &&
} + {error && } + {!loading && !error && !result && ( + + )} + {result && img && ( + <> +
+
+ + {faces.length > 1 && ( +
+ {faces.map((_, i) => ( + + ))} +
+ )} +
+
+ {focus && ( + <> +
+
+

Face {focusIdx + 1}

+ {antiSpoofing && } +
+
+ {focus.age != null && <>
Age
~{Math.round(focus.age)}
} + {focus.dominant_gender && <>
Gender
{focus.dominant_gender}
} + {focus.dominant_emotion && <>
Emotion
{focus.dominant_emotion}
} + {focus.dominant_race && <>
Race
{focus.dominant_race}
} + {focus.face_confidence != null && <>
Detection
{(focus.face_confidence * 100).toFixed(1)}%
} +
+
+ + + + + )} +
+
+ + + )} +
+
+ ) + } + + function EmbedTab({ model }) { + const [img, setImg] = useState(null) + const [loading, setLoading] = useState(false) + const [error, setError] = useState(null) + const [result, setResult] = useState(null) + const [elapsedMs, setElapsedMs] = useState(null) + + const submit = async (e) => { + e.preventDefault() + if (!model) { ctx.toast('Select a face model first', 'warning'); return } + if (!img) { ctx.toast('Add an image', 'warning'); return } + setLoading(true); setError(null); setResult(null) + const started = performance.now() + try { + const data = await api.face.embed({ model, img: img.dataUrl }) + setElapsedMs(performance.now() - started) + setResult(data) + } catch (err) { + setError(err.message) + } finally { + setLoading(false) + } + } + + return ( +
+ +
+ {loading &&
} + {error && } + {!loading && !error && !result && ( + + )} + {result && ( + + )} +
+
+ ) + } + + function LivenessPill({ isReal, score }) { + if (isReal == null) { + return Not checked + } + return ( + + + {isReal ? 'Real' : 'Spoof'} + {score != null && {(score * 100).toFixed(0)}%} + + ) + } + + // Not ui.EmptyState: this is the biometrics empty card of the core page. + function EmptyState({ icon, title, body }) { + return ( +
+
+ ) + } + + function ResponseDetails({ data }) { + return ( +
+ Raw response + 
{JSON.stringify(data, null, 2)}
+
+ ) + } + + return { MoreFaceTools } +} diff --git a/addons/face/ui/src/api.js b/addons/face/ui/src/api.js new file mode 100644 index 000000000..52ca36227 --- /dev/null +++ b/addons/face/ui/src/api.js @@ -0,0 +1,35 @@ +// The core routes the Face add-on uses, reached with the signed-in user's own +// session (sdk.core.fetch). Paths and bodies are the ones of faceApi in +// core/http/react-ui/src/utils/api.js; only the transport changed. register +// takes { model, name, img, labels }, identify { model, img, ... }. +import { createIdentityApi } from '../../../internal/identity-ui/src/registry.js' + +export function createApi(ctx) { + const { json } = ctx + const query = (params = {}) => { + const q = new URLSearchParams() + for (const [k, v] of Object.entries(params)) if (v !== undefined && v !== null && v !== '') q.set(k, String(v)) + const s = q.toString() + return s ? `?${s}` : '' + } + // /v1/voice/* and /v1/face/*: the shared views read both kinds; the page only + // uses the face one. A failed call keeps its status (a 404 on forget reads as + // already gone). + const identity = createIdentityApi(ctx) + return { + identity, + face: identity.face, + models: { + listCapabilities: () => json('/api/models/capabilities'), + // The gallery list, as modelsApi.list builds it: empty parameters are left out. + list: (params) => json(`/api/models${query(params)}`), + // Starts a gallery install (modelsApi.install). + install: (id) => json(`/api/models/install/${encodeURIComponent(id)}`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: '{}' }), + }, + // The running installs, polled while one is in progress. + operations: { + list: () => json('/api/operations'), + }, + raw: ctx.core, + } +} diff --git a/addons/face/ui/src/ctx.jsx b/addons/face/ui/src/ctx.jsx new file mode 100644 index 000000000..92c318f7f --- /dev/null +++ b/addons/face/ui/src/ctx.jsx @@ -0,0 +1,67 @@ +import en from './i18n/en.js' +import { modelFrom as pickModel } from './lib/model.js' + +// The one object every page factory of the Face add-on receives. It binds the +// SDK once: React, the shared components, the session-bound core fetch, the +// translation and the small adapters that replace the host router and toasts. +export function createContext(sdk) { + const { React } = sdk + const h = React.createElement + const Fragment = React.Fragment + sdk.locale.define('en', Object.fromEntries(Object.entries(en).filter(([, v]) => typeof v === 'string'))) + const lists = Object.fromEntries(Object.entries(en).filter(([, v]) => Array.isArray(v))) + + const t = (key, vars) => sdk.locale.t(key, key, vars) + // A list of strings or objects (the "why" and "stored" notes). + const list = (key) => lists[key] || [] + + // A link to a page of this add-on, as an anchor the shell navigates. + function Link({ page, query, className, children, ...rest }) { + const href = sdk.navigation.href(query ? `${page}?${query}` : page) + return h('a', { + href, className, + onClick: (e) => { if (e.metaKey || e.ctrlKey || e.shiftKey || e.button !== 0) return; e.preventDefault(); sdk.navigation.navigate(query ? `${page}?${query}` : page) }, + ...rest, + }, children) + } + + const useSearch = () => new URLSearchParams(sdk.navigation.current().search) + // The model a page works on: ?model= first (a moved link carries it + // there, because a model id may hold "/" or ":" that a path segment cannot), + // then the first path segment of a link typed by hand. + const modelFrom = (path) => pickModel(sdk.navigation.current().search, path) + + const toast = (message, kind = 'info') => (sdk.notify[kind] || sdk.notify.info)(message) + + // The path prefix LocalAI is served under, read the way the host reads it + // (utils/basePath.js: the element). document.baseURI would be the + // address of the current page when there is no such element. + const basePath = (() => { + if (typeof document === 'undefined') return '' + const el = document.querySelector('base[href]') + if (!el) return '' + try { return new URL(el.getAttribute('href')).pathname.replace(/\/+$/, '') } catch { return '' } + })() + + const json = async (path, init) => { + const response = await sdk.core.fetch(path, init) + if (!response.ok) { + const body = await response.json().catch(() => ({})) + throw new Error(body?.error?.message || `HTTP ${response.status}`) + } + return response.json() + } + + // The undo bar of a delayed action: text and an Undo button. Replaces the + // host's toast with an undo, which the SDK does not expose. + function UndoBar({ text, undoLabel, onUndo }) { + return h('div', { className: 'vc-undo', role: 'status' }, h('span', null, text), h(sdk.ui.Button, { variant: 'ghost', size: 'sm', onClick: onUndo }, undoLabel)) + } + + return { + sdk, React, h, Fragment, t, list, Icon: sdk.Icon, ui: sdk.ui, hooks: sdk.hooks, viewer: sdk.viewer, core: sdk.core, + Link, useSearch, modelFrom, toast, json, UndoBar, basePath, + openPanel: (title, render) => sdk.layers.openPanel({ title, render }), + markDirty: (flag) => sdk.markDirty(flag), + } +} diff --git a/addons/face/ui/src/i18n/en.js b/addons/face/ui/src/i18n/en.js new file mode 100644 index 000000000..efa468ebb --- /dev/null +++ b/addons/face/ui/src/i18n/en.js @@ -0,0 +1,198 @@ +// Generated once from the host locale files; edit by hand from here on. +export default { + "faces.title": "Faces", + "faces.lede": "Find out whose face is in a photo, or whether two photos show the same person. Photos go only to the model running on this server.", + "tool.aria": "Recognition tool", + "tool.sameTab": "Same person?", + "tool.clip": "Clip", + "tool.clipA": "Clip A", + "tool.clipB": "Clip B", + "tool.photo": "Photo", + "tool.photoA": "Photo A", + "tool.photoB": "Photo B", + "tool.compare": "Compare", + "tool.comparing": "Comparing", + "tool.workingAria": "Working on it", + "face.person": "person", + "face.print": "faceprint", + "face.whoTab": "Who is this", + "face.whoLede": "Matches a photo against the people you enrolled.", + "face.sameLede": "Two photos, one question. Nobody has to be enrolled.", + "face.whoGo": "Find who this is", + "face.working": "Searching", + "face.whoNote": "Searches the people enrolled on this server. The photo you test with is not stored.", + "face.sameNote": "The answer comes from one faceprint per photo. Nothing is stored.", + "face.workingText": "Comparing the photo with the faceprints the server holds.", + "face.probably": "Probably {{name}}", + "face.matchText": "{{name}}'s faceprint is the closest of {{count}} compared, {{distance}} away. Anything under {{cutoff}} counts as the same person.", + "face.nobody": "Nobody enrolled looks like this", + "face.missText": "The closest was {{name}} at {{distance}}, past the cut-off of {{cutoff}}. That usually means a new person, or a photo that is blurred, turned away or in poor light.", + "face.noneEnrolled": "Nobody is enrolled to compare with", + "face.noneEnrolledText": "Enrol a person first, then look up a photo against them.", + "face.noneReturned": "The server returned no people", + "face.noneReturnedText": "It holds no faceprints made with this model. If the server restarted, enrolled faces are gone: enrol them again.", + "face.enrolThis": "Enrol this face", + "face.same": "Probably the same person", + "face.different": "Probably different people", + "face.knownTitle": "Known people", + "face.enrol": "Enrol a person", + "face.enrolTitle": "Enrol a person", + "face.enrolIntro": "The server keeps a faceprint and a name. It does not keep the photo.", + "face.emptyTitle": "Nobody enrolled in this browser yet", + "face.emptyText": "Enrol a person with one clear photo, then test a photo against them.", + "face.agreedHint": "I have their permission, or another lawful basis, to store a faceprint of them.", + "face.keepHint": "Lets you see the photo and enrol again after a server restart. Off by default. It is stored in this browser only.", + "face.antiSpoof": "Check for photos of photos", + "face.antiSpoofHint": "Needs a model that supports it.", + "face.real": "Real", + "face.spoof": "Spoof", + "face.found": "Face in photo {{n}}", + "face.neededTitle": "Faces need a face model", + "face.neededText": "Face recognition turns a photo into a faceprint and compares faceprints. It runs on a face-recognition model, and none is installed on this server.", + "face.neededWhat": "Install a model that supports face recognition. Once one is installed, this page turns on by itself. Users also need the Face recognition permission.", + "face.moreTitle": "More tools", + "face.moreHint": "detect faces, guess traits, get the raw faceprint", + "face.why": [ + "The server matches one face per photo. A group photo is matched on a single face.", + "Light, angle, glasses and blur change a face a lot.", + "This is a similarity score, not a security check. Do not use it to unlock anything." + ], + "face.stored": [ + { + "where": "On the server", + "what": "A {{print}} (numbers that describe the face, not the picture), a name and your labels. They are kept in memory and are lost when the server restarts. The photo is not kept." + }, + { + "where": "In this browser", + "what": "The name, labels and time of each person you enrolled. A copy of the photo only if you tick Keep a copy when enrolling." + }, + { + "where": "Photos you test with", + "what": "They go to the model on this server to be compared and are not kept." + } + ], + "same.text": "The two {{print}}s are {{distance}} apart. Anything under {{cutoff}} counts as the same person.", + "same.distance": "distance", + "word.strong": "Strong match", + "word.likely": "Likely match", + "word.close": "Close call", + "word.near": "Just past the cut-off", + "scale.aria": "Distance scale with the cut-off at {{cutoff}}. Lower means more alike. {{dots}}", + "scale.cutoff": "cut-off {{value}}", + "scale.axis": "distance between {{print}}s, lower means more alike", + "scale.oneDot": "one dot per enrolled {{person}}", + "scale.onePair": "one dot for the pair", + "ranks.aria": "Closest people, nearest first", + "ranks.within": "within cut-off", + "ranks.far": "too far", + "cutoff.label": "Cut-off", + "cutoff.default": "model default", + "cutoff.reset": "Reset to {{value}}", + "cutoff.stricter": "Stricter", + "cutoff.looser": "Looser", + "cutoff.explain": "Stricter means fewer wrong matches and more misses. The answer above updates as you move it. The server is not asked again.", + "how.title": "How to read this.", + "how.body": "Distance is how far apart two {{print}}s are, 0 meaning identical. The cut-off is the line between same and different. The word Strong or Likely says how far inside the line a result sits. It is not a probability.", + "why.title": "Why this might be wrong", + "why.short": "The clip is short: {{s}} s. Three seconds or more of speech works better.", + "clip.mic": "microphone", + "clip.camera": "camera", + "clip.record": "Record", + "clip.takePhoto": "Take a photo", + "clip.chooseAudio": "Choose a file", + "clip.choosePhoto": "Choose a photo", + "clip.readAloud": "Read this aloud, normal pace", + "clip.remove": "Remove", + "clip.blocked": "The {{thing}} is blocked. Allow it in the browser's site settings and try again, or choose a file instead.", + "clip.insecure": "A browser only opens the {{thing}} on https or on localhost. You are on {{origin}}. Open this page through localhost or add TLS, or choose a file.", + "clip.unsupported": "This browser cannot open the {{thing}}. Choose a file instead.", + "clip.missing": "No {{thing}} was found on this device. Choose a file instead.", + "clip.failed": "The {{thing}} could not be opened. Choose a file instead.", + "clip.allowed": "Microphone allowed.", + "clip.capture": "Capture", + "clip.cancel": "Cancel", + "clip.recording": "Recording, {{s}} s", + "clip.micReady": "Microphone ready", + "clip.stop": "Stop", + "clip.start": "Start recording", + "clip.recorded": "Recording", + "clip.photographed": "Photo", + "clip.uploaded": "Uploaded file", + "clip.goodLength": "good length", + "clip.short": "short", + "clip.checkLevel": "check level", + "clip.preview": "Preview", + "clip.photoAlt": "The photo you chose", + "known.caption": "People saved in this browser, with their {{noun}} state", + "known.name": "Name", + "known.added": "Added", + "known.actions": "Actions", + "known.best": "best match", + "known.notOnServer": "not on the server", + "known.notInBrowser": "not in this browser's list", + "known.removing": "removing", + "known.goneText": "The last search did not return this person. The server may have restarted, so the {{print}} is gone, or it was enrolled with another model.", + "known.extraText": "The server returned this person but this browser has no record of them.", + "known.noLabels": "no labels", + "known.undo": "Undo", + "known.reenrolSaved": "Re-enrol from saved copy", + "known.enrolAgain": "Enrol again", + "known.play": "Play the saved sample of {{name}}", + "known.forget": "Remove {{name}}", + "known.note": "This list is kept in this browser. The server has no list call, so it cannot be checked until you run a search.", + "known.noteSearched": "This list is kept in this browser. The last search checked it against the server; a person it did not return is marked.", + "stored.title": "What is stored, and where", + "enrol.close": "Close", + "enrol.cancel": "Cancel", + "enrol.submit": "Enrol", + "enrol.working": "Enrolling", + "enrol.ready": "Ready to enrol.", + "enrol.stillNeeded": "Still needed: {{list}}.", + "enrol.needModel": "a model", + "enrol.needRecording": "a recording", + "enrol.needPhoto": "a photo", + "enrol.needName": "a name", + "enrol.needTranscript": "a transcript", + "enrol.needPermission": "permission", + "enrol.recordingStep": "Recording", + "enrol.photoStep": "Photo", + "enrol.recordingLabel": "Recording", + "enrol.photoLabel": "Photo", + "enrol.whoStep": "Who is it", + "enrol.name": "Name", + "enrol.namePlaceholder": "e.g. Marta Rossi", + "enrol.labels": "Labels", + "enrol.labelsHint": "optional, one per line as key: value", + "enrol.asSpeech": "Also keep it as a speech voice", + "enrol.asSpeechHint": "Text-to-speech can then speak in this voice. Needs the exact transcript.", + "enrol.transcript": "Exact transcript", + "enrol.permissionStep": "Permission", + "enrol.agreed": "This person agreed", + "enrol.keep": "Keep a copy in this browser", + "toast.enrolled": "Enrolled {{name}}", + "toast.enrolledBoth": "Enrolled {{name}} and saved a speech voice", + "toast.tooBig": "The copy was too large for browser storage, so it was not kept.", + "toast.speechFailed": "Enrolled, but the speech voice could not be saved: {{message}}", + "toast.removing": "{{name}} will be removed.", + "toast.removed": "Removed {{name}}", + "toast.alreadyGone": "{{name}} was already gone from the server.", + "error.retry": "Try again", + "error.traces": "Open traces", + "needed.badge": "Not enabled on this server", + "needed.backend": "backend {{backend}}", + "needed.install": "Install", + "needed.installing": "Installing", + "needed.started": "Installing {{name}}. Progress is in Operations.", + "needed.loadFailed": "Models could not be listed.", + "needed.noneListed": "No matching model is listed in the galleries.", + "needed.browse": "Browse all models", + "needed.askAdmin": "Ask an administrator to install one.", + "needed.whatTitle": "What turns this on", + "off.badge": "Not enabled for your account", + "off.voice.title": "Voice recognition is off for your account", + "off.voice.text": "An administrator has not allowed you to use voice recognition.", + "off.voice.what": "An administrator can turn on the Voice recognition permission for your user in Users and keys.", + "off.face.title": "Face recognition is off for your account", + "off.face.text": "An administrator has not allowed you to use face recognition.", + "off.face.what": "An administrator can turn on the Face recognition permission for your user in Users and keys." +} diff --git a/addons/face/ui/src/index.jsx b/addons/face/ui/src/index.jsx new file mode 100644 index 000000000..c6cd5f677 --- /dev/null +++ b/addons/face/ui/src/index.jsx @@ -0,0 +1,27 @@ +import { createContext } from './ctx.jsx' +import { createApi } from './api.js' +import { createFaceRecognition } from './FaceRecognition.jsx' +import * as identity from '../../../internal/identity-ui/src/identity.js' + +// The second argument is for tests: the host passes the SDK only. +export function createAddon(sdk, options = {}) { + const ctx = createContext(sdk) + // The JSX of every factory compiles to calls of `h`; ctx binds it once and + // each factory reads it from ctx. This `h` is for the JSX in this file. + // eslint-disable-next-line no-unused-vars + const { h } = ctx + const api = createApi(ctx) + const pages = { + recognition: createFaceRecognition(ctx, api, identity).FaceRecognition, + ...options.pages, + } + + function Component({ page, path }) { + const Page = pages[page] + return Page ? : null + } + return { Component, components: {}, chatActions: [], dispose() {} } +} + +// For tests: the context of an SDK, built the way createAddon builds it. +export const __ctxFor = (sdk) => createContext(sdk) diff --git a/addons/face/ui/src/lib/model.js b/addons/face/ui/src/lib/model.js new file mode 100644 index 000000000..1482007d9 --- /dev/null +++ b/addons/face/ui/src/lib/model.js @@ -0,0 +1,8 @@ +// The model a page works on. A moved link carries it as ?model=, because +// a model id may hold "/" or ":" that an add-on path segment refuses; a link +// typed by hand may still carry it as the first path segment. +export function modelFrom(search, path) { + const fromQuery = new URLSearchParams(search || '').get('model') + if (fromQuery) return fromQuery + return String(path || '').split('/')[0] +} diff --git a/addons/face/ui/src/style.css b/addons/face/ui/src/style.css new file mode 100644 index 000000000..b5c764bb3 --- /dev/null +++ b/addons/face/ui/src/style.css @@ -0,0 +1,242 @@ +.vc-undo { display: flex; gap: var(--dk-space-2, 8px); align-items: center; } +.vc-field { display: flex; flex-direction: column; gap: var(--dk-space-1, 4px); margin-bottom: var(--dk-space-3, 12px); } +.vc-note { color: var(--dk-text-muted, var(--dk-text)); } + +/* ========================================================================== + Voices and Faces. Built on the kit's table, sheet, hub tab, range, field, + chip, badge and toast classes. What is here is what the kit does not say: + the distance scale, the verdict, the clip card, the registry's person cell + and the speech voice rows. Colours, radii and spacing are kit roles (--dk-*). + Surfaces are flat and nothing sits on a coloured rail at the edge. Motion + is switched off for reduced motion at the end of this file. + ========================================================================== */ + +.idn-page { display: flex; flex-direction: column; gap: var(--dk-space-5); min-width: 0; } +.idn-page :is(a, button, summary, select, input, textarea, [tabindex]):focus-visible { outline: 2px solid var(--dk-accent-text); outline-offset: 2px; } +.idn-h2 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-17) / 1.3 var(--dk-font-display); color: var(--dk-text); } +.idn-page > .page-header { margin-bottom: 0; } +.idn-modelpick { min-width: 240px; max-width: 100%; } +.idn-model { display: inline-flex; align-items: center; gap: var(--dk-space-2); width: 100%; height: 36px; padding: 0 var(--dk-space-3); border: 1px solid var(--dk-control-edge); border-radius: var(--dk-radius-2); background: var(--dk-card); color: var(--dk-text); font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-mono); cursor: pointer; } +.idn-model:hover:not(:disabled) { background: var(--dk-hover); } +.idn-model:disabled { opacity: .5; cursor: not-allowed; } +.idn-model .searchable-select__label { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; text-align: left; } +.idn-model .searchable-select__chevron { color: var(--dk-muted); font-size: 10px; } +.idn-tabs { align-self: stretch; max-width: 100%; } +.idn-tabs .dk-tab { text-decoration: none; display: inline-flex; align-items: center; gap: var(--dk-space-2); } +.idn-tabs .dk-tab[aria-selected="true"]::after { transform: scaleX(1); } + +/* ---- The tool card -------------------------------------------------------- */ +.idn-bench { display: flex; flex-direction: column; gap: var(--dk-space-5); min-width: 0; } +.idn-tool { padding: var(--dk-space-4) var(--dk-space-5) var(--dk-space-5); display: flex; flex-direction: column; gap: var(--dk-space-4); min-width: 0; } +.idn-tool__bar { display: flex; align-items: center; gap: var(--dk-space-3); flex-wrap: wrap; } +.idn-tool__lede { margin: 0; } +.idn-tool__body { display: flex; flex-direction: column; gap: var(--dk-space-4); min-width: 0; } +.idn-tool__go { display: flex; align-items: center; gap: var(--dk-space-3); flex-wrap: wrap; } +.idn-tool__option { align-items: flex-start; } +.idn-pair { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: var(--dk-space-3); } +.idn-working { display: flex; flex-direction: column; gap: var(--dk-space-2); } +.idn-working .dk-hint { margin: 0; } + +/* ---- Clip input ----------------------------------------------------------- */ +.idn-clip { display: flex; flex-direction: column; gap: var(--dk-space-3); padding: var(--dk-space-3) var(--dk-space-4); border-radius: var(--dk-radius-3); background: var(--dk-inset); border: 1px solid var(--dk-edge); min-width: 0; } +.idn-clip__head { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-2); min-height: 28px; } +.idn-clip__choices { display: flex; flex-direction: column; gap: var(--dk-space-3); align-items: flex-start; } +.idn-clip__buttons { display: flex; align-items: center; gap: var(--dk-space-2); flex-wrap: wrap; } +.idn-clip__prompt { margin: 0; padding: var(--dk-space-3); border-radius: var(--dk-radius-2); background: var(--dk-card); border: 1px solid var(--dk-edge); font-size: var(--dk-text-15); line-height: 1.5; } +.idn-clip__prompt .dk-eyebrow { display: block; margin-bottom: 4px; } +.idn-clip__hint, .idn-clip__notice { display: flex; align-items: flex-start; gap: var(--dk-space-2); margin: 0; font-size: var(--dk-text-13); color: var(--dk-muted); } +.idn-clip__hint .lai-icon, .idn-clip__notice .lai-icon { flex: none; margin-top: 2px; } +.idn-clip__notice { padding: var(--dk-space-2) var(--dk-space-3); border-radius: var(--dk-radius-2); background: var(--dk-card); border: 1px solid var(--dk-edge); color: var(--dk-text); } +.idn-clip__notice[role="alert"] { background: var(--dk-warn-wash); border-color: transparent; color: var(--dk-warn-text); } +.idn-clip__live { display: flex; flex-direction: column; gap: var(--dk-space-3); } +.idn-clip__video { width: 100%; max-width: 420px; border-radius: var(--dk-radius-2); background: var(--dk-canvas); } +.idn-clip__rec { margin: 0; display: flex; align-items: center; gap: var(--dk-space-2); font-size: var(--dk-text-15); } +.idn-clip__card { display: flex; flex-direction: column; gap: var(--dk-space-3); min-width: 0; } +.idn-clip__meta { display: flex; align-items: center; gap: var(--dk-space-2); flex-wrap: wrap; min-width: 0; } +.idn-clip__name { min-width: 0; overflow-wrap: anywhere; font-size: var(--dk-text-15); } +.idn-clip__img { max-width: 100%; max-height: 260px; width: auto; border-radius: var(--dk-radius-2); object-fit: contain; align-self: flex-start; background: var(--dk-canvas); } +.idn-clip__notes { margin: 0; padding-left: 18px; font-size: var(--dk-text-13); color: var(--dk-warn-text); } +.idn-clip .voice-profile-player, .idn-clip .waveform-player { min-width: 0; } + +/* ---- Result: verdict, scale, ranks, cut-off ------------------------------- */ +.idn-result { display: flex; flex-direction: column; gap: var(--dk-space-4); padding-top: var(--dk-space-4); border-top: 1px solid var(--dk-edge); min-width: 0; } +.idn-verdict { display: grid; grid-template-columns: 44px minmax(0, 1fr); gap: var(--dk-space-3); align-items: start; } +.idn-verdict__icon { display: grid; place-items: center; width: 44px; height: 44px; border-radius: 50%; background: var(--dk-inset); color: var(--dk-muted); font-size: 20px; } +.idn-verdict[data-tone="match"] .idn-verdict__icon { background: var(--dk-ok-wash); color: var(--dk-ok-text); } +.idn-verdict__title { margin: 0; display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-2); font: var(--dk-weight-bold) var(--dk-text-21) / 1.25 var(--dk-font-display); overflow-wrap: anywhere; } +.idn-verdict__title .dk-badge { font-family: var(--dk-font-body); } +.idn-verdict__text { margin: 4px 0 0; max-width: 70ch; font-size: var(--dk-text-15); line-height: 1.5; } + +.idn-scale { margin: 0; padding: 0 36px; } +.idn-scale__plot { position: relative; height: calc(108px + var(--rows, 0) * 18px); } +.idn-scale__track { position: absolute; left: 0; right: 0; top: 38px; height: 8px; border-radius: var(--dk-radius-pill); background: var(--dk-inset); overflow: hidden; border: 1px solid var(--dk-edge); box-sizing: border-box; } +.idn-scale__inside { position: absolute; left: 0; top: 0; bottom: 0; width: var(--at); background: var(--dk-accent-wash); } +.idn-scale__cut { position: absolute; left: var(--at); top: 22px; height: 32px; width: 2px; margin-left: -1px; background: var(--dk-text); z-index: 1; } +.idn-scale__cut-label { position: absolute; bottom: 100%; left: 50%; transform: translateX(-50%); white-space: nowrap; padding-bottom: 2px; font-size: 11px; font-weight: 600; } +.idn-scale__dot { position: absolute; left: var(--at); top: 0; width: 0; height: 0; z-index: 2; } +.idn-scale__mark { position: absolute; left: -8px; top: 34px; width: 16px; height: 16px; box-sizing: border-box; border-radius: 50%; background: var(--dk-card); border: 2px solid var(--dk-control-edge); } +.idn-scale__dot[data-within="true"] .idn-scale__mark { border-color: var(--dk-accent-text); } +.idn-scale__dot[data-best="true"] .idn-scale__mark { background: var(--dk-accent); border-color: var(--dk-accent); box-shadow: 0 0 0 3px var(--dk-card); } +.idn-scale__name { position: absolute; left: 0; transform: translateX(-50%); top: 58px; max-width: 120px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-size: 12px; } +.idn-scale__dot[data-row="1"] .idn-scale__name { top: 76px; } +.idn-scale__dot[data-row="2"] .idn-scale__name { top: 94px; } +.idn-scale__ticks { position: absolute; left: 0; right: 0; top: calc(86px + var(--rows, 0) * 18px); height: 16px; } +.idn-scale__tick { position: absolute; left: var(--at); transform: translateX(-50%); font-size: 11px; color: var(--dk-muted); } +.idn-scale__tick::before { content: ""; position: absolute; left: 50%; bottom: 100%; height: 5px; width: 1px; background: var(--dk-control-edge); margin-bottom: 1px; } +.idn-scale__caption { display: flex; justify-content: space-between; gap: var(--dk-space-3); margin: var(--dk-space-2) -36px 0; padding: 0 4px; font-size: 12px; color: var(--dk-muted); } + +.idn-ranks { margin: 0; padding: 0; list-style: none; } +.idn-ranks li { display: grid; grid-template-columns: 32px minmax(0, 1fr) auto auto; align-items: center; gap: var(--dk-space-3); min-height: 40px; border-bottom: 1px solid var(--dk-edge); font-size: var(--dk-text-15); } +.idn-ranks li:last-child { border-bottom: 0; } +.idn-ranks li[data-within="true"] .idn-ranks__name { font-weight: var(--dk-weight-semibold); } +.idn-ranks__n { color: var(--dk-muted); font-size: 12px; } +.idn-ranks__name { min-width: 0; overflow-wrap: anywhere; } + +.idn-result__tune { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1.2fr); gap: var(--dk-space-5); align-items: start; } +.idn-cutoff { display: flex; flex-direction: column; gap: var(--dk-space-2); min-width: 0; } +.idn-cutoff__head { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-2); flex-wrap: wrap; } +.idn-cutoff__row { display: grid; grid-template-columns: auto minmax(0, 1fr) auto; align-items: center; gap: var(--dk-space-3); } +.idn-cutoff .dk-hint, .idn-how { margin: 0; font-size: var(--dk-text-13); color: var(--dk-muted); line-height: 1.5; } +.idn-how strong { color: var(--dk-text); } +.idn-result__enrol { align-self: flex-start; } +.idn-face { margin: 0; display: flex; flex-direction: column; gap: var(--dk-space-2); } + +/* ---- Disclosures ---------------------------------------------------------- */ +.idn-disclosure > summary { display: flex; align-items: center; gap: var(--dk-space-2); min-height: 40px; cursor: pointer; font-weight: var(--dk-weight-semibold); font-size: var(--dk-text-15); list-style: none; } +.idn-disclosure > summary::-webkit-details-marker { display: none; } +.idn-disclosure > summary .lai-icon { color: var(--dk-muted); transition: transform var(--dk-dur-1) var(--dk-ease-out); } +.idn-disclosure[open] > summary .lai-icon:first-child { transform: rotate(90deg); } +.idn-disclosure > summary .dk-hint { font-weight: 400; } +.idn-disclosure__list { margin: 0 0 var(--dk-space-3); padding-left: 38px; font-size: var(--dk-text-13); color: var(--dk-muted); line-height: 1.6; } +.idn-disclosure__text { margin: 0 0 var(--dk-space-3); padding-left: 24px; font-size: var(--dk-text-13); color: var(--dk-muted); max-width: 70ch; } +.idn-disclosure--page { border-top: 1px solid var(--dk-edge); } +.idn-stored { margin: 0 0 var(--dk-space-3); padding-left: 24px; display: grid; gap: var(--dk-space-3); } +.idn-stored div { display: grid; grid-template-columns: 160px minmax(0, 1fr); gap: var(--dk-space-3); } +.idn-stored dt { font-weight: var(--dk-weight-semibold); font-size: var(--dk-text-13); } +.idn-stored dd { margin: 0; font-size: var(--dk-text-13); color: var(--dk-muted); line-height: 1.55; max-width: 70ch; } +.idn-more__body { padding: 0 0 var(--dk-space-4) 24px; } + +/* ---- Known people --------------------------------------------------------- */ +.idn-known { display: flex; flex-direction: column; gap: var(--dk-space-2); min-width: 0; } +.idn-known__head { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-3); flex-wrap: wrap; } +.idn-known__note { margin: 0; } +.idn-known .dk-table-wrap { border: 1px solid var(--dk-edge); border-radius: var(--dk-radius-3); background: var(--dk-card); } +.idn-known__empty { padding: var(--dk-space-6) var(--dk-space-4); border: 1px solid var(--dk-edge); border-radius: var(--dk-radius-3); background: var(--dk-card); } +.idn-who { display: flex; align-items: center; gap: var(--dk-space-3); min-width: 0; } +.idn-who__text { display: flex; flex-direction: column; min-width: 0; } +.idn-who .dk-table-name { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-2); } +.idn-avatar { display: inline-grid; place-items: center; flex: none; width: 36px; height: 36px; border-radius: 50%; overflow: hidden; background: var(--dk-inset); border: 1px solid var(--dk-edge); font: var(--dk-weight-bold) 12px var(--dk-font-mono); color: var(--dk-text); } +.idn-avatar img { width: 100%; height: 100%; object-fit: cover; } +.idn-avatar[data-tone="warn"] { background: var(--dk-warn-wash); color: var(--dk-warn-text); border-color: transparent; } +.idn-when { white-space: nowrap; font-size: 12px; color: var(--dk-muted); } +.idn-actions { text-align: right; white-space: nowrap; } +.idn-actions > * { vertical-align: middle; } +.idn-known tr[data-pending="true"] .idn-who { opacity: .55; } + +/* ---- Enrol sheet ---------------------------------------------------------- */ +.idn-sheet { display: flex; flex-direction: column; gap: var(--dk-space-5); } +.idn-sheet__step { display: flex; flex-direction: column; gap: var(--dk-space-3); } +.idn-sheet__h { margin: 0; display: flex; align-items: center; gap: var(--dk-space-2); font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); } +.idn-sheet__n { display: inline-grid; place-items: center; width: 24px; height: 24px; border-radius: 50%; border: 1.5px solid var(--dk-accent-text); color: var(--dk-accent-text); font: 600 12px var(--dk-font-mono); } +.idn-sheet__choice { align-items: flex-start; gap: var(--dk-space-3); } +.idn-sheet__choice > span { display: flex; flex-direction: column; gap: 2px; } +.idn-sheet__choice .dk-hint { margin: 0; } +.idn-sheet__need { margin-right: auto; font-size: var(--dk-text-13); color: var(--dk-muted); } + +/* ---- Errors, disabled state, recording tab -------------------------------- */ +.idn-error { display: grid; grid-template-columns: 32px minmax(0, 1fr); gap: var(--dk-space-3); padding: var(--dk-space-3) var(--dk-space-4); border-radius: var(--dk-radius-2); background: var(--dk-error-wash); color: var(--dk-error-text); } +.idn-error__icon { font-size: 22px; padding-top: 2px; } +.idn-error__title { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); } +.idn-error__text, .idn-error__raw { margin: 4px 0 0; font-size: var(--dk-text-13); } +.idn-error__raw { opacity: .85; overflow-wrap: anywhere; } +.idn-error__acts { display: flex; gap: var(--dk-space-2); flex-wrap: wrap; margin-top: var(--dk-space-3); } +.idn-error .dk-btn { color: inherit; } +.idn-needed { padding: var(--dk-space-5); display: flex; flex-direction: column; align-items: flex-start; gap: var(--dk-space-3); } +.idn-needed__text { margin: 0; max-width: 62ch; font-size: var(--dk-text-15); line-height: 1.5; } +.idn-needed__list { margin: 0; padding: 0; list-style: none; width: min(100%, 560px); border: 1px solid var(--dk-edge); border-radius: var(--dk-radius-3); } +.idn-needed__list li { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-3); padding: var(--dk-space-3) var(--dk-space-4); border-bottom: 1px solid var(--dk-edge); } +.idn-needed__list li:last-child { border-bottom: 0; } +.idn-needed__list li > span { display: flex; flex-direction: column; min-width: 0; } +.idn-needed__list .dk-hint { margin: 0; } +.idn-off { min-height: 40vh; } +.idn-recording { padding: var(--dk-space-5); display: flex; flex-direction: column; align-items: flex-start; gap: var(--dk-space-3); } +.idn-recording__text { margin: 0; max-width: 62ch; line-height: 1.5; } +.idn-recording__steps { margin: 0; padding-left: 20px; display: grid; gap: 4px; color: var(--dk-muted); font-size: var(--dk-text-15); } +.idn-recording .dk-hint { margin: 0; max-width: 62ch; } + +/* ---- Speech voices -------------------------------------------------------- */ +.voice-library-page { display: flex; flex-direction: column; gap: var(--dk-space-4); min-width: 0; } +.idn-speech__head { display: flex; align-items: flex-start; justify-content: space-between; gap: var(--dk-space-4); flex-wrap: wrap; } +.idn-speech__lede { margin: 4px 0 0; max-width: 56ch; } +.idn-speech__acts { display: flex; align-items: center; gap: var(--dk-space-3); flex-wrap: wrap; } +.voice-library-toolbar { display: flex; gap: var(--dk-space-3); flex-wrap: wrap; } +.voice-library-search { flex: 1 1 260px; min-width: 0; } +.voice-library-language { min-width: 160px; } +.voice-library-loading { display: flex; align-items: center; gap: var(--dk-space-3); padding: var(--dk-space-5); color: var(--dk-muted); } +.idn-speech__list { border: 1px solid var(--dk-edge); border-radius: var(--dk-radius-3); background: var(--dk-card); overflow: hidden; } +.idn-speech__list .dk-row { text-align: left; } +.voice-row__end { display: flex; align-items: center; gap: var(--dk-space-3); } +.voice-row__meta { font-size: 12px; color: var(--dk-muted); white-space: nowrap; } +.voice-row__waveform { display: inline-flex; align-items: center; gap: 2px; width: 120px; height: 22px; } +.voice-row__waveform > span { flex: 1; min-width: 2px; border-radius: 2px; background: var(--dk-control-edge); } +.voice-row__chevron { color: var(--dk-muted); } +.voice-library-empty { min-height: 0; } +.voice-library-detail { display: flex; flex-direction: column; gap: var(--dk-space-4); min-width: 0; } +.voice-detail__player { min-width: 0; } +.voice-detail__section { display: flex; flex-direction: column; gap: var(--dk-space-2); } +.voice-detail__section h3 { margin: 0; } +.voice-detail__section blockquote { margin: 0; padding: var(--dk-space-3) var(--dk-space-4); border-radius: var(--dk-radius-2); background: var(--dk-inset); border: 1px solid var(--dk-edge); font-size: var(--dk-text-15); line-height: 1.55; overflow-wrap: anywhere; } +.voice-detail__metadata { margin: 0; } +.voice-detail__model-setup { display: flex; flex-direction: column; gap: var(--dk-space-3); padding: var(--dk-space-4); border-radius: var(--dk-radius-3); background: var(--dk-inset); border: 1px solid var(--dk-edge); } +.voice-detail__model-setup-heading { display: flex; gap: var(--dk-space-3); align-items: flex-start; } +.voice-detail__model-setup-heading h3 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); } +.voice-detail__model-setup-heading p, .voice-detail__model-fallback, .voice-detail__capability-note { margin: 4px 0 0; font-size: var(--dk-text-13); color: var(--dk-muted); } +.voice-detail__model-list { margin: 0; padding: 0; list-style: none; display: grid; gap: var(--dk-space-2); } +.voice-detail__model-list li { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-3); } +.voice-detail__model-list li > span { display: flex; flex-direction: column; min-width: 0; } +.voice-detail__model-list small { color: var(--dk-muted); } +.voice-detail__model-loading { display: flex; align-items: center; gap: var(--dk-space-2); font-size: var(--dk-text-13); color: var(--dk-muted); } +.voice-detail__model-fallback a { color: var(--dk-accent-text); } +.voice-detail__api { border-top: 1px solid var(--dk-edge); } +.voice-detail__api-body { display: flex; flex-direction: column; gap: var(--dk-space-3); padding: 0 0 var(--dk-space-3) 24px; font-size: var(--dk-text-13); color: var(--dk-muted); } +.voice-detail__api-body > p { margin: 0; } +.voice-detail__compatible-models { display: flex; flex-direction: column; gap: var(--dk-space-2); } +.voice-detail__compatible-models ul { display: flex; flex-wrap: wrap; gap: var(--dk-space-2); margin: 0; padding: 0; list-style: none; } +.voice-detail__compatible-models li { display: inline-flex; align-items: center; gap: var(--dk-space-2); padding: 2px 10px; border-radius: var(--dk-radius-pill); background: var(--dk-ok-wash); color: var(--dk-ok-text); } +.voice-detail__code-heading { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-2); } +.voice-detail__api pre { margin: 0; padding: var(--dk-space-3); overflow-x: auto; border-radius: var(--dk-radius-2); background: var(--dk-inset); border: 1px solid var(--dk-edge); color: var(--dk-text); font-size: 12px; line-height: 1.55; } +.voice-delete { margin-right: auto; } + +/* ---- Phone ---------------------------------------------------------------- */ +@media (max-width: 720px) { + .idn-tool { padding: var(--dk-space-3); } + .idn-pair { grid-template-columns: minmax(0, 1fr); } + .idn-result__tune { grid-template-columns: minmax(0, 1fr); gap: var(--dk-space-4); } + .idn-scale { padding: 0 26px; } + .idn-scale__caption { margin: var(--dk-space-2) -26px 0; flex-direction: column; gap: 2px; } + .idn-stored div { grid-template-columns: minmax(0, 1fr); gap: 2px; } + .idn-verdict { grid-template-columns: 36px minmax(0, 1fr); } + .idn-verdict__icon { width: 36px; height: 36px; font-size: 17px; } + .idn-verdict__title { font-size: var(--dk-text-17); } + .idn-modelpick { min-width: 0; width: 100%; } + .voice-row__waveform { display: none; } + .idn-ranks li { grid-template-columns: 20px minmax(0, 1fr) auto auto; gap: var(--dk-space-2); font-size: var(--dk-text-13); } + .idn-known .dk-table td { padding-inline: var(--dk-space-2); } + .idn-actions { white-space: normal; } + .idn-actions .dk-btn--secondary { margin-bottom: 4px; } + .idn-more__body { padding-left: 0; } + .idn-more > summary .dk-hint { display: none; } + .idn-needed__list li { flex-direction: column; align-items: flex-start; } +} + +@media (prefers-reduced-motion: no-preference) { + .idn-clip__rec .dk-dot[data-live="true"] { animation: idn-pulse 1.1s ease-in-out infinite; } +} +@keyframes idn-pulse { 50% { opacity: .35; } } +@media (prefers-reduced-motion: reduce) { + .idn-disclosure > summary .lai-icon { transition: none; } + /* The kit's indeterminate bar keeps sliding; here it sits still at its width. */ + .idn-working .dk-progress[data-indeterminate] .dk-progress-bar { animation: none; } +} + +.idn-who .dk-table-sub { white-space: normal; overflow: visible; text-overflow: clip; max-width: 62ch; } diff --git a/addons/face/ui/test/model.test.mjs b/addons/face/ui/test/model.test.mjs new file mode 100644 index 000000000..3d863ec38 --- /dev/null +++ b/addons/face/ui/test/model.test.mjs @@ -0,0 +1,17 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { modelFrom } from '../src/lib/model.js' + +test('the model comes from ?model first and keeps slashes, colons and percent signs', () => { + assert.equal(modelFrom('model=org%2Fname%3Av1', 'ignored'), 'org/name:v1') +}) + +test('without ?model the model is the first path segment', () => { + assert.equal(modelFrom('', 'speechbrain/extra'), 'speechbrain') + assert.equal(modelFrom('x=1', 'speechbrain'), 'speechbrain') +}) + +test('no model gives an empty string', () => { + assert.equal(modelFrom('', ''), '') + assert.equal(modelFrom(undefined, undefined), '') +}) diff --git a/addons/face/ui/test/recognition.test.mjs b/addons/face/ui/test/recognition.test.mjs new file mode 100644 index 000000000..701e8d6b1 --- /dev/null +++ b/addons/face/ui/test/recognition.test.mjs @@ -0,0 +1,86 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { createRequire } from 'node:module' +import { fileURLToPath } from 'node:url' +import { dirname, join, resolve } from 'node:path' +import { load } from './render.test.mjs' + +const here = dirname(fileURLToPath(import.meta.url)) +const require = createRequire(join(resolve(here, '../../../../core/http/react-ui'), 'package.json')) +const React = require('react') +const { renderToString } = require('react-dom/server') + +test('recognition: the first paint has the faces page test id', async () => { + const { mod, sdk } = await load() + const addon = await mod.createAddon(sdk) + const html = renderToString(React.createElement(addon.Component, { page: 'recognition', path: '' })) + assert.match(html, /data-testid="faces-page"/) +}) + +test('recognition: a viewer without face_recognition gets the off notice', async () => { + const { mod, sdk } = await load({ viewer: { isAdmin: () => false, hasFeature: (f) => f !== 'face_recognition' } }) + const addon = await mod.createAddon(sdk) + const html = renderToString(React.createElement(addon.Component, { page: 'recognition', path: '' })) + assert.match(html, /data-testid="face-off"/) + assert.doesNotMatch(html, /idn-bench/) +}) + +test('recognition: register, identify and forget use the core face routes', async () => { + const { createApi } = await import('../src/api.js') + const seen = [] + const ctx = { core: { fetch: async (p, i) => { seen.push([p, i?.method]); return { ok: true, json: async () => ({}) } } }, json: async (p, i) => { seen.push([p, i?.method]); return {} } } + const api = createApi(ctx) + await api.face.register({ model: 'm', img: 'x', name: 'a' }) + await api.face.identify({ model: 'm', img: 'x' }) + await api.face.forget({ model: 'm', id: 'a' }) + assert.deepEqual(seen.map(s => s[0]), ['/v1/face/register', '/v1/face/identify', '/v1/face/forget']) + assert.deepEqual(seen.map(s => s[1]), ['POST', 'POST', 'POST']) +}) + +test('recognition: any viewer who holds face_recognition gets the library, administrator or not', async () => { + const { mod, sdk } = await load({ viewer: { isAdmin: () => false, hasFeature: () => true } }) + const addon = await mod.createAddon(sdk) + const html = renderToString(React.createElement(addon.Component, { page: 'recognition', path: '' })) + assert.match(html, /idn-bench/) + assert.doesNotMatch(html, /data-testid="face-off"/) +}) + +test('recognition: the model of the path reaches the model selection', async () => { + const seen = [] + const names = ['Button', 'Spinner', 'EmptyState', 'Badge', 'Stack', 'Text', 'PageHeader', 'ConfirmDialog', 'ErrorWithTraceLink', 'MediaInput', 'WaveformPlayer', 'DistributionBars', 'EmbeddingInspector', 'TabSwitch'] + const components = Object.fromEntries(names.map(n => [n, (p) => React.createElement('div', null, p.title || p.label || p.children || null)])) + components.PageHeader = (p) => React.createElement('header', null, p.title, p.actions) + components.ModelSelector = (p) => { seen.push(p.value); return React.createElement('div') } + const { mod, sdk } = await load({ components }) + const addon = await mod.createAddon(sdk) + renderToString(React.createElement(addon.Component, { page: 'recognition', path: 'insightface' })) + assert.deepEqual(seen, ['insightface']) +}) + +test('recognition: the bundle has the face canvas, the face strings and no host import', async () => { + const { readFileSync } = await import('node:fs') + const { build } = await import('../build.mjs') + const dir = await build() + const js = readFileSync(join(dir, 'ui/face.js'), 'utf8') + assert.match(js, /biometrics-bbox/) + assert.doesNotMatch(js, /from ['"]react(-dom)?/) + assert.match(js, /faces\.title/) +}) + +test('recognition: every translation key the sources read is in the Face bundle', async () => { + const { readFileSync, readdirSync, statSync } = await import('node:fs') + const { default: en } = await import('../src/i18n/en.js') + const roots = [join(here, '../src'), join(here, '../../../internal/identity-ui/src')] + const files = [] + const walk = (d) => { for (const f of readdirSync(d)) { const p = join(d, f); if (statSync(p).isDirectory()) walk(p); else if (/\.jsx?$/.test(p) && !p.includes('i18n')) files.push(p) } } + roots.forEach(walk) + const missing = [] + for (const f of files) { + for (const m of readFileSync(f, 'utf8').matchAll(/\bt\(\s*'([a-zA-Z]+\.[a-zA-Z0-9_.]+)'/g)) { + // The voice branch of a shared view is never reached on this page. + if (m[1].startsWith('voice.')) continue + if (!(m[1] in en)) missing.push(`${f.split('/').pop()}: ${m[1]}`) + } + } + assert.deepEqual(missing, []) +}) diff --git a/addons/face/ui/test/render.test.mjs b/addons/face/ui/test/render.test.mjs new file mode 100644 index 000000000..30eacd508 --- /dev/null +++ b/addons/face/ui/test/render.test.mjs @@ -0,0 +1,75 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { createRequire } from 'node:module' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { dirname, join, resolve } from 'node:path' +import { build } from '../build.mjs' + +const here = dirname(fileURLToPath(import.meta.url)) +const hostUi = resolve(here, '../../../../core/http/react-ui') +const require = createRequire(join(hostUi, 'package.json')) +const React = require('react') +const { renderToString } = require('react-dom/server') + +export async function load(extra = {}) { + const dir = await build() + const mod = await import(pathToFileURL(join(dir, 'ui/face.js')).href + `?t=${Date.now()}`) + const { createHostSDK } = await import(pathToFileURL(join(hostUi, 'src/addons/sdk.js')).href) + const stub = (tag) => (p) => React.createElement(tag, { 'data-stub': tag }, p.title || p.label || p.children || null) + const calls = [] + const controller = new AbortController() + const sdk = createHostSDK({ + addonId: 'face', generation: 1, signal: controller.signal, pages: ['recognition'], + host: { + React, + components: Object.fromEntries(['Button', 'Spinner', 'EmptyState', 'Badge', 'Stack', 'Text', 'PageHeader', 'ConfirmDialog', 'ErrorWithTraceLink', 'ModelSelector', 'MediaInput', 'WaveformPlayer', 'DistributionBars', 'EmbeddingInspector', 'TabSwitch'].map(n => [n, n === 'PageHeader' ? (p => React.createElement('header', { 'data-stub': 'header' }, p.title ? React.createElement('h1', null, p.title) : null, p.supporting)) : stub('div')])), + hooks: { useMediaCapture: () => ({}), useObjectUrl: () => null, useDelayedAction: () => ({ pending: null, schedule() {}, undo() {} }) }, + viewer: { isAdmin: () => true, hasFeature: () => true }, + basePath: '', + fetch: async (url, init) => { calls.push([url, init]); return { ok: true, json: async () => ({ data: [] }), blob: async () => new Blob([]) } }, + Icon: () => null, theme: { mode: () => 'light', subscribe: () => () => {} }, language: () => 'en', subscribeLanguage: () => () => {}, api: {}, + ...extra, + }, + bindings: { current: () => ({ page: 'recognition', path: '', search: '' }), openLayer: () => ({ close() {} }), notify() {} }, + }) + return { mod, sdk, controller, calls } +} + +test('render: createAddon returns a page component and dispose', async () => { + const { mod, sdk, controller } = await load() + const addon = await mod.createAddon(sdk) + assert.equal(typeof addon.Component, 'function') + assert.equal(typeof addon.dispose, 'function') + addon.dispose() + controller.abort('done') +}) + +test('render: an unknown page draws nothing', async () => { + const { mod, sdk } = await load() + const addon = await mod.createAddon(sdk) + assert.equal(renderToString(React.createElement(addon.Component, { page: 'nope', path: '' })), '') +}) + +test('render: the manifest declares the recognition page, the main page and SDK 2', async () => { + const { readFileSync } = await import('node:fs') + const m = JSON.parse(readFileSync(join(here, '../../manifest.json'), 'utf8')) + assert.equal(m.id, 'face') + assert.deepEqual(m.capabilities.ui.pages, ['recognition']) + assert.deepEqual(m.capabilities.ui.navigation, []) + assert.equal(m.capabilities.ui.mainPage, 'recognition') + assert.deepEqual(m.capabilities.ui.sdk, { min: 2, max: 2 }) + assert.deepEqual(m.capabilities.tools ?? [], []) +}) + +test('render: a registered page is drawn with its path', async () => { + const { mod, sdk } = await load() + const addon = await mod.createAddon(sdk, { pages: { recognition: ({ path }) => React.createElement('p', null, `page:${path}`) } }) + assert.equal(renderToString(React.createElement(addon.Component, { page: 'recognition', path: 'abc' })), '

page:abc

') +}) + +test('render: the recognition page draws exactly one h1, its own title', async () => { + const { mod, sdk } = await load() + const addon = await mod.createAddon(sdk) + const html = renderToString(React.createElement(addon.Component, { page: 'recognition', path: '' })) + assert.deepEqual([...html.matchAll(/]*>(.*?)<\/h1>/g)].map(m => m[1]), ['Faces']) +}) diff --git a/addons/face/ui/test/run.mjs b/addons/face/ui/test/run.mjs new file mode 100644 index 000000000..f95b03d47 --- /dev/null +++ b/addons/face/ui/test/run.mjs @@ -0,0 +1,23 @@ +// Runs the node unit tests of the Voice UI package and prints one JSON line +// per test, the same shape as the host's runner (core/http/react-ui/tests/unit): +// {"case":"unit: ","status":"passed"|"failed"|"skipped"} +import { run } from 'node:test' +import { readdirSync } from 'node:fs' +import { dirname, join } from 'node:path' +import { fileURLToPath } from 'node:url' + +const here = dirname(fileURLToPath(import.meta.url)) +const names = readdirSync(here).filter(f => f.endsWith('.test.mjs')).sort() +let failed = false +const stream = run({ files: names.map(f => join(here, f)), concurrency: 1 }) +stream.on('test:pass', ({ name, skip, todo, nesting }) => { + if (nesting > 0) return + console.log(JSON.stringify({ case: `unit: ${name}`, status: skip || todo ? 'skipped' : 'passed' })) +}) +stream.on('test:fail', ({ name, nesting, details }) => { + if (nesting > 0) return + failed = true + console.log(JSON.stringify({ case: `unit: ${name}`, status: 'failed', error: String(details && details.error && details.error.message || 'failed').slice(0, 400) })) +}) +stream.on('end', () => { process.exit(failed ? 1 : 0) }) +stream.resume() diff --git a/addons/fine-tuning/cmd/fine-tuning/main.go b/addons/fine-tuning/cmd/fine-tuning/main.go new file mode 100644 index 000000000..782e810ba --- /dev/null +++ b/addons/fine-tuning/cmd/fine-tuning/main.go @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: MIT + +// Command fine-tuning is the executable of the fine-tuning add-on. It serves +// the typed tools of the add-on and reaches the job engine through the host +// operations; the add-on UI works in the browser through the SDK. +package main + +import ( + "os" + + "github.com/mudler/LocalAI/addons/fine-tuning/internal/tools" + "github.com/mudler/LocalAI/addons/internal/hostclient" + "github.com/mudler/LocalAI/addons/internal/protocol" + "github.com/mudler/LocalAI/pkg/addons/contract" +) + +func main() { + os.Exit(protocol.Run(os.Args, func(s protocol.Startup) contract.Runtime { + return tools.New(hostclient.New(s), s.DataDir) + })) +} diff --git a/addons/fine-tuning/internal/tools/journal.go b/addons/fine-tuning/internal/tools/journal.go new file mode 100644 index 000000000..cb9897b7a --- /dev/null +++ b/addons/fine-tuning/internal/tools/journal.go @@ -0,0 +1,142 @@ +// SPDX-License-Identifier: MIT +package tools + +import ( + "encoding/json" + "errors" + "os" + "path/filepath" + "sync" + + "github.com/mudler/LocalAI/pkg/addons/contract" +) + +// maxReceipts bounds the receipts the journal keeps. A repeat of an older +// operation is not answered from memory any more; the host asks for a receipt +// only for a recent, unsettled operation. +const maxReceipts = 1024 + +// job is what the add-on knows of one resource a tool created: a job it +// started or a dataset it sent. The engine job ID +// never leaves the add-on: a model names the job by its resource. +type job struct { + JobID string `json:"jobId,omitempty"` + Name string `json:"name"` + // Dataset marks a resource that stands for an inline dataset and not for a + // job; Ref is the reference the host issued for it. + Dataset bool `json:"dataset,omitempty"` + Ref string `json:"ref,omitempty"` + // Revision counts the changes a tool made to the job. It is the data + // revision of the resource, so a stale approval is noticed. + Revision uint64 `json:"revision"` +} + +type journalFile struct { + Jobs map[string]job `json:"jobs"` + Receipts map[string]contract.Receipt `json:"receipts"` + Order []string `json:"order"` +} + +// journal maps resource IDs to engine job IDs and keeps the receipts of the +// operations that changed something. It is one JSON file in the data +// directory, written through a temporary file and renamed over the old one. +type journal struct { + mu sync.Mutex + path string + data journalFile +} + +func openJournal(dir string) *journal { + j := &journal{path: filepath.Join(dir, "journal.json")} + j.data = journalFile{Jobs: map[string]job{}, Receipts: map[string]contract.Receipt{}} + if b, err := os.ReadFile(j.path); err == nil { + var f journalFile + if json.Unmarshal(b, &f) == nil { + if f.Jobs != nil { + j.data.Jobs = f.Jobs + } + if f.Receipts != nil { + j.data.Receipts = f.Receipts + } + j.data.Order = f.Order + } + } + return j +} + +func (j *journal) job(resource string) (job, bool) { + j.mu.Lock() + defer j.mu.Unlock() + v, ok := j.data.Jobs[resource] + return v, ok +} + +func (j *journal) receipt(op string) (contract.Receipt, bool) { + j.mu.Lock() + defer j.mu.Unlock() + r, ok := j.data.Receipts[op] + return r, ok +} + +// commit stores the receipt of an operation and, when upd is not nil, the new +// state of a job in one write. A failed write leaves the memory as it was, so +// the caller reports the failure and nothing claims a state that is not on disk. +func (j *journal) commit(resource string, upd *job, r contract.Receipt) error { + j.mu.Lock() + defer j.mu.Unlock() + next := journalFile{Jobs: map[string]job{}, Receipts: map[string]contract.Receipt{}, Order: append([]string(nil), j.data.Order...)} + for k, v := range j.data.Jobs { + next.Jobs[k] = v + } + for k, v := range j.data.Receipts { + next.Receipts[k] = v + } + if upd != nil { + next.Jobs[resource] = *upd + } + if _, seen := next.Receipts[r.OperationID]; !seen { + next.Order = append(next.Order, r.OperationID) + } + next.Receipts[r.OperationID] = r + for len(next.Order) > maxReceipts { + delete(next.Receipts, next.Order[0]) + next.Order = next.Order[1:] + } + if err := writeAtomic(j.path, next); err != nil { + return err + } + j.data = next + return nil +} + +func writeAtomic(path string, v any) error { + b, err := json.Marshal(v) + if err != nil { + return err + } + tmp, err := os.CreateTemp(filepath.Dir(path), "journal-*.tmp") + if err != nil { + return err + } + name := tmp.Name() + fail := func(e error) error { _ = tmp.Close(); _ = os.Remove(name); return e } + if _, err := tmp.Write(b); err != nil { + return fail(err) + } + if err := tmp.Sync(); err != nil { + return fail(err) + } + if err := tmp.Close(); err != nil { + _ = os.Remove(name) + return err + } + if err := os.Chmod(name, 0o600); err != nil { + _ = os.Remove(name) + return errors.Join(err) + } + if err := os.Rename(name, path); err != nil { + _ = os.Remove(name) + return err + } + return nil +} diff --git a/addons/fine-tuning/internal/tools/suite_test.go b/addons/fine-tuning/internal/tools/suite_test.go new file mode 100644 index 000000000..ab3dabb08 --- /dev/null +++ b/addons/fine-tuning/internal/tools/suite_test.go @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT +package tools + +import ( + "testing" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +func TestTools(t *testing.T) { + RegisterFailHandler(Fail) + RunSpecs(t, "Fine-tuning tools suite") +} diff --git a/addons/fine-tuning/internal/tools/tools.go b/addons/fine-tuning/internal/tools/tools.go new file mode 100644 index 000000000..f83573ca4 --- /dev/null +++ b/addons/fine-tuning/internal/tools/tools.go @@ -0,0 +1,557 @@ +// SPDX-License-Identifier: MIT + +// Package tools is the runtime of the fine-tuning add-on executable. A job that +// a tool starts is a resource of the host: the host reserves the resource ID, +// and this package remembers which engine job it stands for. A tool can only +// control a job that a tool started, because only those have a resource. +package tools + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "regexp" + "strings" + "time" + "unicode/utf8" + + "github.com/mudler/LocalAI/addons/internal/hostclient" + "github.com/mudler/LocalAI/addons/internal/protocol" + "github.com/mudler/LocalAI/pkg/addons/contract" +) + +// Tool names. They are the entries of capabilities.tools in the manifest and +// the exact values of arguments.tool. +const ( + ToolResourceExists = "resource-exists" + ToolList = "fine-tune-list" + ToolStart = "fine-tune-start" + ToolDataset = "fine-tune-dataset" + ToolStatus = "fine-tune-status" + ToolCheckpoints = "fine-tune-checkpoints" + ToolStop = "fine-tune-stop" + ToolExport = "fine-tune-export" +) + +// Tools is the declared tool set of this add-on, in manifest order. +// +// fine-tune-list is a read tool that names no resource: it lists the jobs of the +// calling user, the ones started in the UI included. The host runs it as that +// user and the host operation scopes the answer to the same user. Every other +// tool works on a resource that a tool created. +var Tools = []string{ToolResourceExists, ToolList, ToolStart, ToolDataset, ToolStatus, ToolCheckpoints, ToolStop, ToolExport} + +// Engine is the host operation channel. *hostclient.Client implements it, and +// a nil client answers hostclient.ErrNoHostOps. +type Engine interface { + Do(ctx context.Context, operationID, op string, args, out any) error +} + +var _ Engine = (*hostclient.Client)(nil) + +// schema is the schema revision of every job. The data revision counts the +// changes a tool made. +const schema uint64 = 1 + +var ( + modelName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`) + exportFormats = map[string]bool{"lora": true, "merged_16bit": true, "merged_4bit": true, "gguf": true} +) + +const maxNameRunes = 80 + +// maxDatasetBytes is the bound of an inline dataset, the one of the host +// operation. +const maxDatasetBytes = 900 << 10 + +var datasetExt = []string{".jsonl", ".json", ".csv", ".txt"} + +type runtime struct { + eng Engine + j *journal + now func() time.Time +} + +// New returns the runtime of the executable. dataDir holds the journal. +func New(eng Engine, dataDir string) contract.Runtime { + return &runtime{eng: eng, j: openJournal(dataDir), now: time.Now} +} + +// startArgs are the arguments of fine-tune-start. The host adds "tool"; the +// name is the label the host shows in the approval card and the resource list. +type startArgs struct { + Tool string `json:"tool"` + Name string `json:"name"` + Model string `json:"model"` + Backend string `json:"backend"` + TrainingType string `json:"trainingType"` + TrainingMethod string `json:"trainingMethod"` + DatasetSource string `json:"datasetSource"` + DatasetSplit string `json:"datasetSplit"` + LearningRate float64 `json:"learningRate"` + NumEpochs int64 `json:"numEpochs"` + BatchSize int64 `json:"batchSize"` + MaxSteps int64 `json:"maxSteps"` + AdapterRank int64 `json:"adapterRank"` + AdapterAlpha int64 `json:"adapterAlpha"` +} + +// engineStart is the argument object of finetune.start. It has exactly the +// fields the host operation accepts and none of the add-on's own. +type engineStart struct { + Model string `json:"model"` + Backend string `json:"backend,omitempty"` + TrainingType string `json:"trainingType,omitempty"` + TrainingMethod string `json:"trainingMethod,omitempty"` + DatasetSource string `json:"datasetSource"` + DatasetSplit string `json:"datasetSplit,omitempty"` + LearningRate float64 `json:"learningRate,omitempty"` + NumEpochs int64 `json:"numEpochs,omitempty"` + BatchSize int64 `json:"batchSize,omitempty"` + MaxSteps int64 `json:"maxSteps,omitempty"` + AdapterRank int64 `json:"adapterRank,omitempty"` + AdapterAlpha int64 `json:"adapterAlpha,omitempty"` +} + +// datasetArgs are the arguments of fine-tune-dataset. The name is the file +// name of the dataset and the label of the resource. +type datasetArgs struct { + Tool string `json:"tool"` + Name string `json:"name"` + Content string `json:"content"` +} + +type stopArgs struct { + Tool string `json:"tool"` + SaveCheckpoint bool `json:"saveCheckpoint"` +} + +type exportArgs struct { + Tool string `json:"tool"` + ExportFormat string `json:"exportFormat"` + Name string `json:"name"` +} + +// decode reads the arguments into a closed struct: an unknown field or +// trailing data is a bad request. +func decode(raw json.RawMessage, v any) error { + if len(raw) == 0 { + raw = json.RawMessage(`{}`) + } + dec := json.NewDecoder(bytes.NewReader(raw)) + dec.DisallowUnknownFields() + if err := dec.Decode(v); err != nil { + return protocol.ErrBadRequest + } + if _, err := dec.Token(); err != io.EOF { + return protocol.ErrBadRequest + } + return nil +} + +func toolOf(env contract.Envelope) (string, error) { + var a struct { + Tool string `json:"tool"` + } + if json.Unmarshal(env.Request.Arguments, &a) != nil || a.Tool == "" { + return "", protocol.ErrBadRequest + } + return a.Tool, nil +} + +func payload(v any) json.RawMessage { b, _ := json.Marshal(v); return b } + +// mapErr turns the refusal of a host operation into the protocol error that +// tells the owner what happened. A failure that does not say the request was +// wrong stays as it is, and the owner reads it as uncertain. +func mapErr(err error) error { + var he *contract.HostError + if !errors.As(err, &he) { + return err + } + switch he.Code { + case contract.HostBadRequest, contract.HostTooLarge: + return fmt.Errorf("%w: %s", protocol.ErrBadRequest, he.Message) + case contract.HostNotFound: + return fmt.Errorf("%w: %s", protocol.ErrNotFound, he.Message) + case contract.HostConflict: + return fmt.Errorf("%w: %s", protocol.ErrConflict, he.Message) + case contract.HostFeature, contract.HostScope: + return fmt.Errorf("%w: %s", protocol.ErrForbidden, he.Message) + } + return err +} + +// jobOf is the job of the resource. A dataset resource is not a job. +func (r *runtime) jobOf(env contract.Envelope) (job, error) { + j, ok := r.entryOf(env) + if !ok || j.Dataset { + return job{}, protocol.ErrNotFound + } + return j, nil +} + +func (r *runtime) entryOf(env contract.Envelope) (job, bool) { + if env.Request.ResourceID == "" { + return job{}, false + } + return r.j.job(env.Request.ResourceID) +} + +func (r *runtime) Read(ctx context.Context, env contract.Envelope) (contract.Result, error) { + tool, err := toolOf(env) + if err != nil { + return contract.Result{}, err + } + op := env.Request.OperationID + switch tool { + case ToolResourceExists: + j, ok := r.j.job(env.Request.ResourceID) + if !ok { + return contract.Result{Payload: payload(map[string]bool{"exists": false})}, nil + } + if j.Dataset { + // The reference lives in the host and the journal; there is no + // engine job to ask. + return contract.Result{Payload: payload(map[string]bool{"exists": true})}, nil + } + err := r.eng.Do(ctx, op, "finetune.get", map[string]string{"jobId": j.JobID}, nil) + var he *contract.HostError + switch { + case err == nil: + return contract.Result{Payload: payload(map[string]bool{"exists": true})}, nil + case errors.As(err, &he) && he.Code == contract.HostNotFound: + return contract.Result{Payload: payload(map[string]bool{"exists": false})}, nil + } + // The engine could not answer. The host settles a creation only on an + // explicit answer, so this is an error and never "does not exist". + return contract.Result{}, err + case ToolList: + // No resource: the host runs this read as the calling user and the + // host operation lists only that user's jobs. + var a struct { + Tool string `json:"tool"` + } + if err := decode(env.Request.Arguments, &a); err != nil { + return contract.Result{}, err + } + var got json.RawMessage + if err := r.eng.Do(ctx, op, "finetune.list", nil, &got); err != nil { + return contract.Result{}, mapErr(err) + } + return contract.Result{Payload: got}, nil + case ToolStatus: + j, ok := r.entryOf(env) + if !ok { + return contract.Result{}, protocol.ErrNotFound + } + if j.Dataset { + return contract.Result{Payload: payload(map[string]any{ + "name": j.Name, "revision": contract.Revision{Schema: schema, Data: j.Revision}, "kind": "dataset", "datasetRef": j.Ref, + })}, nil + } + var got map[string]any + if err := r.eng.Do(ctx, op, "finetune.get", map[string]string{"jobId": j.JobID}, &got); err != nil { + return contract.Result{}, mapErr(err) + } + // The engine job ID stays in the add-on. The name and the revision + // are what the host reads to label and track the resource. + delete(got, "id") + got["name"] = j.Name + got["revision"] = contract.Revision{Schema: schema, Data: j.Revision} + return contract.Result{Payload: payload(got)}, nil + case ToolCheckpoints: + j, err := r.jobOf(env) + if err != nil { + return contract.Result{}, err + } + var got json.RawMessage + if err := r.eng.Do(ctx, op, "finetune.checkpoints", map[string]string{"jobId": j.JobID}, &got); err != nil { + return contract.Result{}, mapErr(err) + } + return contract.Result{Payload: got}, nil + } + return contract.Result{}, protocol.ErrBadRequest +} + +func (r *runtime) checkStart(a startArgs) error { + if a.Model == "" || a.DatasetSource == "" || len([]rune(a.Name)) > maxNameRunes || + a.LearningRate < 0 || a.NumEpochs < 0 || a.BatchSize < 0 || a.MaxSteps < 0 || a.AdapterRank < 0 || a.AdapterAlpha < 0 { + return protocol.ErrBadRequest + } + return nil +} + +func checkExport(a exportArgs) error { + if a.ExportFormat != "" && !exportFormats[a.ExportFormat] { + return protocol.ErrBadRequest + } + if a.Name != "" && !modelName.MatchString(a.Name) { + return protocol.ErrBadRequest + } + return nil +} + +func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Preview, error) { + tool, err := toolOf(env) + if err != nil { + return contract.Preview{}, err + } + out := contract.Preview{OperationID: env.Request.OperationID, ExpiresAt: r.now().Add(time.Minute)} + switch tool { + case ToolStart: + var a startArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return out, err + } + if err := r.checkStart(a); err != nil { + return out, err + } + // The resource does not exist yet; an operation that already + // committed is replayed by Apply, so a repeat is not a clash. + if _, taken := r.j.job(env.Request.ResourceID); taken { + if _, done := r.j.receipt(env.Request.OperationID); !done { + return out, protocol.ErrConflict + } + } + out.Revision = contract.Revision{Schema: schema, Data: 0} + out.Summary = payload(map[string]any{"count": 1, "sample": map[string]string{"action": "start fine-tuning job", "name": nameOf(a), "model": a.Model}}) + return out, nil + case ToolDataset: + var a datasetArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return out, err + } + if err := checkDataset(a); err != nil { + return out, err + } + if _, taken := r.j.job(env.Request.ResourceID); taken { + if _, done := r.j.receipt(env.Request.OperationID); !done { + return out, protocol.ErrConflict + } + } + out.Revision = contract.Revision{Schema: schema, Data: 0} + // The card shows the name and the size, never the content. + out.Summary = payload(map[string]any{"count": 1, "sample": map[string]any{"action": "send dataset", "name": a.Name, "bytes": len(a.Content)}}) + return out, nil + case ToolStop, ToolExport: + j, err := r.jobOf(env) + if err != nil { + return out, err + } + sample := map[string]any{"name": j.Name} + if tool == ToolStop { + var a stopArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return out, err + } + sample["action"], sample["saveCheckpoint"] = "stop fine-tuning job", a.SaveCheckpoint + } else { + var a exportArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return out, err + } + if err := checkExport(a); err != nil { + return out, err + } + sample["action"], sample["exportFormat"] = "export fine-tuned model", a.ExportFormat + } + out.Revision = contract.Revision{Schema: schema, Data: j.Revision} + out.Summary = payload(map[string]any{"count": 1, "sample": sample}) + return out, nil + } + return out, protocol.ErrBadRequest +} + +// checkDataset applies the rules of the host operation early, so a card is not +// shown for a dataset the host would refuse. +func checkDataset(a datasetArgs) error { + if a.Content == "" || len(a.Content) > maxDatasetBytes || !utf8.ValidString(a.Content) || len(a.Name) == 0 || len(a.Name) > 255 || strings.ContainsAny(a.Name, `/\`) || a.Name == "." || a.Name == ".." { + return protocol.ErrBadRequest + } + for _, ext := range datasetExt { + if strings.HasSuffix(a.Name, ext) && len(a.Name) > len(ext) { + return nil + } + } + return protocol.ErrBadRequest +} + +func nameOf(a startArgs) string { + if a.Name != "" { + return a.Name + } + return a.Model +} + +func (r *runtime) Apply(ctx context.Context, env contract.Envelope, rev contract.Revision) (contract.Receipt, error) { + tool, err := toolOf(env) + if err != nil { + return contract.Receipt{}, err + } + // An operation that already committed is answered from its receipt: the + // engine ran once and runs no more. + if rec, ok := r.j.receipt(env.Request.OperationID); ok { + return rec, nil + } + switch tool { + case ToolStart: + return r.applyStart(ctx, env) + case ToolDataset: + return r.applyDataset(ctx, env) + case ToolStop: + var a stopArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return contract.Receipt{}, err + } + return r.mutate(ctx, env, rev, "finetune.stop", func(id string) any { + return map[string]any{"jobId": id, "saveCheckpoint": a.SaveCheckpoint} + }) + case ToolExport: + var a exportArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return contract.Receipt{}, err + } + if err := checkExport(a); err != nil { + return contract.Receipt{}, err + } + return r.mutate(ctx, env, rev, "finetune.export", func(id string) any { + args := map[string]any{"jobId": id} + if a.ExportFormat != "" { + args["exportFormat"] = a.ExportFormat + } + if a.Name != "" { + args["name"] = a.Name + } + return args + }) + } + return contract.Receipt{}, protocol.ErrBadRequest +} + +func (r *runtime) applyStart(ctx context.Context, env contract.Envelope) (contract.Receipt, error) { + var a startArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return contract.Receipt{}, err + } + if err := r.checkStart(a); err != nil { + return contract.Receipt{}, err + } + res, op := env.Request.ResourceID, env.Request.OperationID + if res == "" { + return contract.Receipt{}, protocol.ErrBadRequest + } + if _, taken := r.j.job(res); taken { + return contract.Receipt{}, protocol.ErrConflict + } + var started struct { + ID string `json:"id"` + Status string `json:"status"` + Message string `json:"message"` + } + err := r.eng.Do(ctx, op, "finetune.start", engineStart{ + Model: a.Model, Backend: a.Backend, TrainingType: a.TrainingType, TrainingMethod: a.TrainingMethod, + DatasetSource: a.DatasetSource, DatasetSplit: a.DatasetSplit, LearningRate: a.LearningRate, + NumEpochs: a.NumEpochs, BatchSize: a.BatchSize, MaxSteps: a.MaxSteps, AdapterRank: a.AdapterRank, AdapterAlpha: a.AdapterAlpha, + }, &started) + if err != nil { + return contract.Receipt{}, mapErr(err) + } + if started.ID == "" { + return contract.Receipt{}, errors.New("the engine started a job and returned no ID") + } + rec := contract.Receipt{ + OperationID: op, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: schema, Data: 1}, + Result: contract.Result{Payload: payload(map[string]string{"status": started.Status, "message": started.Message})}, + } + // If this write fails the job runs without a mapping. The error is + // uncertain, so the host asks again with the same operation ID and the + // engine answers it with the same job. + if err := r.j.commit(res, &job{JobID: started.ID, Name: nameOf(a), Revision: 1}, rec); err != nil { + return contract.Receipt{}, err + } + return rec, nil +} + +// applyDataset sends the inline dataset through the host operation, which +// returns a reference only the calling user can use, and remembers it for the +// resource. The reference is in the receipt the person sees and in the status +// of the resource, which only the owner of the resource can read. +func (r *runtime) applyDataset(ctx context.Context, env contract.Envelope) (contract.Receipt, error) { + var a datasetArgs + if err := decode(env.Request.Arguments, &a); err != nil { + return contract.Receipt{}, err + } + if err := checkDataset(a); err != nil { + return contract.Receipt{}, err + } + res, op := env.Request.ResourceID, env.Request.OperationID + if res == "" { + return contract.Receipt{}, protocol.ErrBadRequest + } + if _, taken := r.j.job(res); taken { + return contract.Receipt{}, protocol.ErrConflict + } + var stored struct { + DatasetRef string `json:"datasetRef"` + } + if err := r.eng.Do(ctx, op, "finetune.dataset", map[string]string{"name": a.Name, "content": a.Content}, &stored); err != nil { + return contract.Receipt{}, mapErr(err) + } + if stored.DatasetRef == "" { + return contract.Receipt{}, errors.New("the engine stored a dataset and returned no reference") + } + rec := contract.Receipt{ + OperationID: op, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: schema, Data: 1}, + Result: contract.Result{Payload: payload(map[string]string{"datasetRef": stored.DatasetRef})}, + } + if err := r.j.commit(res, &job{Name: a.Name, Dataset: true, Ref: stored.DatasetRef, Revision: 1}, rec); err != nil { + return contract.Receipt{}, err + } + return rec, nil +} + +// mutate runs a host operation on the job of the resource, once, if the +// resource is still at the revision the person approved. +func (r *runtime) mutate(ctx context.Context, env contract.Envelope, rev contract.Revision, op string, args func(jobID string) any) (contract.Receipt, error) { + j, err := r.jobOf(env) + if err != nil { + return contract.Receipt{}, err + } + id := env.Request.OperationID + if rev.Schema != schema || rev.Data != j.Revision { + return contract.Receipt{OperationID: id, State: contract.ReceiptRejected, + Result: contract.Result{Payload: payload(map[string]any{"error": "stale", "revision": contract.Revision{Schema: schema, Data: j.Revision}})}}, nil + } + var out json.RawMessage + if err := r.eng.Do(ctx, id, op, args(j.JobID), &out); err != nil { + return contract.Receipt{}, mapErr(err) + } + next := j + next.Revision++ + rec := contract.Receipt{OperationID: id, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: schema, Data: next.Revision}} + if len(out) > 0 && json.Valid(out) { + rec.Result.Payload = out + } else { + rec.Result.Payload = payload(map[string]any{}) + } + if err := r.j.commit(env.Request.ResourceID, &next, rec); err != nil { + return contract.Receipt{}, err + } + return rec, nil +} + +func (r *runtime) Receipt(_ context.Context, op string) (contract.Receipt, error) { + if rec, ok := r.j.receipt(op); ok { + return rec, nil + } + return contract.Receipt{}, protocol.ErrNotFound +} + +// OpenOutput refuses: no tool of this add-on produces a file. +func (r *runtime) OpenOutput(context.Context, contract.Envelope, string) (io.ReadCloser, error) { + return nil, protocol.ErrNotFound +} diff --git a/addons/fine-tuning/internal/tools/tools_test.go b/addons/fine-tuning/internal/tools/tools_test.go new file mode 100644 index 000000000..475ce594a --- /dev/null +++ b/addons/fine-tuning/internal/tools/tools_test.go @@ -0,0 +1,527 @@ +// SPDX-License-Identifier: MIT +package tools + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "strings" + "sync" + + "time" + + "github.com/mudler/LocalAI/addons/internal/hostclient" + "github.com/mudler/LocalAI/addons/internal/protocol" + "github.com/mudler/LocalAI/pkg/addons/contract" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +var ctx = context.Background() + +type call struct { + opID, op string + args map[string]any +} + +// fakeEngine records host operations and answers from canned results. A +// result of type error is returned as the failure of that operation. +type fakeEngine struct { + mu sync.Mutex + result map[string]any + calls []call +} + +func (f *fakeEngine) Do(_ context.Context, opID, op string, args, out any) error { + f.mu.Lock() + defer f.mu.Unlock() + var a map[string]any + if args != nil { + b, _ := json.Marshal(args) + _ = json.Unmarshal(b, &a) + } + f.calls = append(f.calls, call{opID, op, a}) + r, ok := f.result[op] + if !ok { + r = map[string]any{} + } + if err, isErr := r.(error); isErr { + return err + } + if out != nil { + b, _ := json.Marshal(r) + return json.Unmarshal(b, out) + } + return nil +} + +func (f *fakeEngine) ops() []string { + f.mu.Lock() + defer f.mu.Unlock() + var out []string + for _, c := range f.calls { + out = append(out, c.op) + } + return out +} + +func (f *fakeEngine) count(op string) int { + n := 0 + for _, o := range f.ops() { + if o == op { + n++ + } + } + return n +} + +func (f *fakeEngine) last(op string) call { + f.mu.Lock() + defer f.mu.Unlock() + for i := len(f.calls) - 1; i >= 0; i-- { + if f.calls[i].op == op { + return f.calls[i] + } + } + return call{} +} + +func envelope(tool, opID, resource, args string) contract.Envelope { + m := map[string]any{} + if args != "" { + _ = json.Unmarshal([]byte(args), &m) + } + m["tool"] = tool + raw, _ := json.Marshal(m) + return contract.Envelope{ + Request: contract.Request{AddonID: "fine-tuning", ResourceID: resource, OperationID: opID, Generation: 1, Action: "apply", Arguments: raw}, + Authority: contract.Authority{Actor: contract.Actor{UserID: "u1", CredentialID: "c1", Mode: "named-user"}, Scope: "apply", ExpiresAt: time.Now().Add(time.Minute)}, + } +} + +const startJSON = `{"name":"my run","model":"org/base","datasetSource":"org/data"}` + +// started returns a runtime that already holds the job job-1 for resource res1. +func started() (contract.Runtime, *fakeEngine) { + eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}}} + rt := New(eng, GinkgoT().TempDir()) + rec, err := rt.Apply(ctx, envelope("fine-tune-start", "create-res1", "res1", startJSON), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred(), "setup") + Expect(rec.State).To(Equal(contract.ReceiptCommitted), "setup: %+v", rec) + return rt, eng +} + +func exists(rt contract.Runtime, res string) (bool, error) { + r, err := rt.Read(ctx, envelope("resource-exists", "x-"+res, res, "")) + if err != nil { + return false, err + } + var v struct { + Exists bool `json:"exists"` + } + _ = json.Unmarshal(r.Payload, &v) + return v.Exists, nil +} + +func loadManifest() contract.Manifest { + b, err := os.ReadFile("../../manifest.json") + Expect(err).NotTo(HaveOccurred()) + m, err := contract.ParseManifest(b) + Expect(err).NotTo(HaveOccurred(), "the manifest must satisfy the contract") + return m +} + +const datasetJSON = `{"name":"data.jsonl","content":"{\"text\":\"a\"}\n"}` + +var _ = Describe("fine-tuning tools", func() { + It("start calls the engine once and remembers the job", func() { + eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}}} + rt := New(eng, GinkgoT().TempDir()) + rec, err := rt.Apply(ctx, envelope("fine-tune-start", "op1", "res1", startJSON), contract.Revision{Schema: 1, Data: 0}) + Expect(err).NotTo(HaveOccurred()) + Expect(rec.State).To(Equal(contract.ReceiptCommitted)) + Expect(rec.Revision.Schema).NotTo(BeZero()) + Expect(eng.ops()).To(Equal([]string{"finetune.start"})) + // The same operation again does not start a second job. + again, err := rt.Apply(ctx, envelope("fine-tune-start", "op1", "res1", startJSON), contract.Revision{Schema: 1, Data: 1}) + Expect(err).NotTo(HaveOccurred()) + Expect(again.State).To(Equal(contract.ReceiptCommitted)) + Expect(eng.count("finetune.start")).To(Equal(1)) + }) + + It("start sends only the fields of the engine operation", func() { + eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}}} + rt := New(eng, GinkgoT().TempDir()) + args := `{"name":"my run","model":"org/base","datasetSource":"org/data","numEpochs":2,"learningRate":0.0002,"adapterRank":8}` + _, err := rt.Apply(ctx, envelope("fine-tune-start", "op1", "res1", args), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred()) + c := eng.last("finetune.start") + Expect(c.opID).To(Equal("op1"), "the call must carry the operation of the request") + for _, banned := range []string{"tool", "name", "resourceId"} { + Expect(c.args).NotTo(HaveKey(banned), "the engine must not receive %q", banned) + } + Expect(c.args["model"]).To(Equal("org/base")) + Expect(c.args["numEpochs"]).To(Equal(float64(2))) + Expect(c.args["adapterRank"]).To(Equal(float64(8))) + }) + + It("previews a start with a nonzero revision and creates nothing", func() { + eng := &fakeEngine{} + rt := New(eng, GinkgoT().TempDir()) + p, err := rt.Preview(ctx, envelope("fine-tune-start", "op1", "res1", startJSON)) + Expect(err).NotTo(HaveOccurred()) + Expect(p.Revision.Schema).NotTo(BeZero()) + Expect(eng.ops()).To(BeEmpty()) + Expect(json.Valid(p.Summary)).To(BeTrue(), "summary %s", p.Summary) + _, err = rt.Preview(ctx, envelope("fine-tune-start", "op1", "res1", `{"name":"x","model":"","datasetSource":"d"}`)) + Expect(err).To(MatchError(protocol.ErrBadRequest), "an empty model must be refused") + }) + + It("status reads the job of the resource", func() { + rt, eng := started() + eng.result["finetune.get"] = map[string]any{"id": "job-1", "status": "training", "model": "org/base"} + res, err := rt.Read(ctx, envelope("fine-tune-status", "r1", "res1", "")) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.last("finetune.get").args["jobId"]).To(Equal("job-1")) + var v map[string]any + Expect(json.Unmarshal(res.Payload, &v)).To(Succeed()) + Expect(v["name"]).To(Equal("my run")) + Expect(v["status"]).To(Equal("training")) + rev, _ := v["revision"].(map[string]any) + Expect(rev["schema"]).NotTo(BeNil(), "inspect needs a revision: %v", v) + Expect(rev["schema"]).NotTo(Equal(float64(0)), "inspect needs a revision: %v", v) + Expect(v).NotTo(HaveKey("id"), "the engine job ID must not reach the model") + Expect(v).NotTo(HaveKey("progressPercent"), "a snapshot percent is unreliable and must not be reported") + }) + + It("checkpoints read the job of the resource", func() { + rt, eng := started() + eng.result["finetune.checkpoints"] = map[string]any{"checkpoints": []any{map[string]any{"step": 10}}} + res, err := rt.Read(ctx, envelope("fine-tune-checkpoints", "r1", "res1", "")) + Expect(err).NotTo(HaveOccurred()) + Expect(string(res.Payload)).To(ContainSubstring(`"step":10`)) + Expect(eng.last("finetune.checkpoints").args["jobId"]).To(Equal("job-1"), "wrong job") + }) + + It("does not let a tool control a job it did not start", func() { + eng := &fakeEngine{} + rt := New(eng, GinkgoT().TempDir()) + _, err := rt.Apply(ctx, envelope("fine-tune-stop", "op2", "nope", `{}`), contract.Revision{Schema: 1}) + Expect(err).To(MatchError(protocol.ErrNotFound), "stop") + _, err = rt.Apply(ctx, envelope("fine-tune-export", "op3", "nope", `{}`), contract.Revision{Schema: 1}) + Expect(err).To(MatchError(protocol.ErrNotFound), "export") + _, err = rt.Read(ctx, envelope("fine-tune-status", "r1", "nope", "")) + Expect(err).To(MatchError(protocol.ErrNotFound), "status") + _, err = rt.Preview(ctx, envelope("fine-tune-stop", "op2", "nope", `{}`)) + Expect(err).To(MatchError(protocol.ErrNotFound), "preview") + Expect(eng.ops()).To(BeEmpty(), "the engine must not be called") + }) + + It("stop and export use the stored job and advance the revision", func() { + rt, eng := started() + eng.result["finetune.stop"] = map[string]any{} + eng.result["finetune.export"] = map[string]any{"model": "my-model"} + p, err := rt.Preview(ctx, envelope("fine-tune-stop", "op2", "res1", `{"saveCheckpoint":true}`)) + Expect(err).NotTo(HaveOccurred()) + Expect(p.Revision.Schema).NotTo(BeZero()) + rec, err := rt.Apply(ctx, envelope("fine-tune-stop", "op2", "res1", `{"saveCheckpoint":true}`), p.Revision) + Expect(err).NotTo(HaveOccurred()) + Expect(rec.State).To(Equal(contract.ReceiptCommitted)) + c := eng.last("finetune.stop") + Expect(c.args["jobId"]).To(Equal("job-1")) + Expect(c.args["saveCheckpoint"]).To(Equal(true)) + Expect(rec.Revision.Data).To(BeNumerically(">", p.Revision.Data), "the revision must advance") + // A repeat of the same operation is answered from the receipt. + _, err = rt.Apply(ctx, envelope("fine-tune-stop", "op2", "res1", `{"saveCheckpoint":true}`), p.Revision) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.count("finetune.stop")).To(Equal(1)) + // Export names the format and the model, and never a path. + ex, err := rt.Apply(ctx, envelope("fine-tune-export", "op3", "res1", `{"exportFormat":"gguf","name":"my-model"}`), rec.Revision) + Expect(err).NotTo(HaveOccurred()) + Expect(ex.State).To(Equal(contract.ReceiptCommitted)) + Expect(string(ex.Result.Payload)).To(ContainSubstring("my-model")) + a := eng.last("finetune.export").args + Expect(a["jobId"]).To(Equal("job-1")) + Expect(a["exportFormat"]).To(Equal("gguf")) + Expect(a["name"]).To(Equal("my-model")) + }) + + It("rejects a stale revision and does not call the engine", func() { + rt, eng := started() + rec, err := rt.Apply(ctx, envelope("fine-tune-stop", "op2", "res1", `{}`), contract.Revision{Schema: 1, Data: 99}) + Expect(err).NotTo(HaveOccurred()) + Expect(rec.State).To(Equal(contract.ReceiptRejected)) + Expect(eng.count("finetune.stop")).To(BeZero(), "the engine was called") + }) + + Describe("host refusals", func() { + for code, want := range map[string]error{ + contract.HostFeature: protocol.ErrForbidden, contract.HostBadRequest: protocol.ErrBadRequest, + contract.HostNotFound: protocol.ErrNotFound, contract.HostConflict: protocol.ErrConflict, + } { + It("maps "+code+" to a protocol error", func() { + rt, eng := started() + eng.result["finetune.export"] = &contract.HostError{Code: code, Message: "m"} + _, err := rt.Apply(ctx, envelope("fine-tune-export", "op-"+code, "res1", `{}`), contract.Revision{Schema: 1, Data: 1}) + Expect(err).To(MatchError(want)) + }) + } + + It("keeps an unavailable engine uncertain", func() { + rt, eng := started() + eng.result["finetune.export"] = &contract.HostError{Code: contract.HostUnavailable, Message: "m"} + _, err := rt.Apply(ctx, envelope("fine-tune-export", "op-u", "res1", `{}`), contract.Revision{Schema: 1, Data: 1}) + Expect(err).To(HaveOccurred()) + for _, e := range []error{protocol.ErrBadRequest, protocol.ErrForbidden, protocol.ErrNotFound, protocol.ErrConflict} { + Expect(err).NotTo(MatchError(e), "an unavailable engine must stay uncertain") + } + }) + }) + + It("answers resource-exists from the journal and the engine", func() { + rt, eng := started() + eng.result["finetune.get"] = map[string]any{"id": "job-1", "status": "training"} + ok, err := exists(rt, "res1") + Expect(err).NotTo(HaveOccurred(), "mapped") + Expect(ok).To(BeTrue(), "mapped") + ok, err = exists(rt, "other") + Expect(err).NotTo(HaveOccurred(), "unmapped") + Expect(ok).To(BeFalse(), "unmapped") + // The job was deleted in the UI: the engine says not found. + eng.result["finetune.get"] = &contract.HostError{Code: contract.HostNotFound, Message: "gone"} + ok, err = exists(rt, "res1") + Expect(err).NotTo(HaveOccurred(), "deleted job") + Expect(ok).To(BeFalse(), "deleted job") + // The engine cannot answer: unknown, never false. + eng.result["finetune.get"] = &contract.HostError{Code: contract.HostUnavailable, Message: "down"} + _, err = exists(rt, "res1") + Expect(err).To(HaveOccurred(), "an unavailable engine must not read as absent") + eng.result["finetune.get"] = hostclient.ErrNoHostOps + _, err = exists(rt, "res1") + Expect(err).To(HaveOccurred(), "no host endpoint must not read as absent") + }) + + It("keeps the journal across a restart", func() { + dir := GinkgoT().TempDir() + eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}, "finetune.get": map[string]any{"status": "training"}}} + _, err := New(eng, dir).Apply(ctx, envelope("fine-tune-start", "op1", "res1", startJSON), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred()) + again := New(eng, dir) + _, err = again.Read(ctx, envelope("fine-tune-status", "r1", "res1", "")) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.last("finetune.get").args["jobId"]).To(Equal("job-1"), "the restarted runtime lost the job") + // The receipt of the first run answers a repeat after the restart. + _, err = again.Apply(ctx, envelope("fine-tune-start", "op1", "res1", startJSON), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.count("finetune.start")).To(Equal(1)) + r, err := again.Receipt(ctx, "op1") + Expect(err).NotTo(HaveOccurred()) + Expect(r.State).To(Equal(contract.ReceiptCommitted)) + _, err = again.Receipt(ctx, "never") + Expect(err).To(MatchError(protocol.ErrNotFound), "an unknown operation must be not found") + // The journal is a plain file written through a temporary one. + entries, _ := os.ReadDir(dir) + for _, e := range entries { + Expect(e.Name()).NotTo(ContainSubstring(".tmp"), "a temporary file was left") + } + _, err = os.Stat(filepath.Join(dir, "journal.json")) + Expect(err).NotTo(HaveOccurred()) + }) + + It("refuses every tool without a host endpoint", func() { + var nilClient *hostclient.Client + dir := GinkgoT().TempDir() + rt := New(nilClient, dir) + _, err := rt.Apply(ctx, envelope("fine-tune-start", "op1", "res1", startJSON), contract.Revision{Schema: 1}) + Expect(err).To(MatchError(hostclient.ErrNoHostOps), "start") + // Seed a job as an earlier run with an endpoint would have. + seed := New(&fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1"}}}, dir) + _, err = seed.Apply(ctx, envelope("fine-tune-start", "op0", "res1", startJSON), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred()) + rt = New(nilClient, dir) + for _, tool := range []string{"fine-tune-status", "fine-tune-checkpoints", "resource-exists"} { + _, err := rt.Read(ctx, envelope(tool, "r-"+tool, "res1", "")) + Expect(err).To(MatchError(hostclient.ErrNoHostOps), tool) + } + for _, tool := range []string{"fine-tune-stop", "fine-tune-export"} { + _, err := rt.Apply(ctx, envelope(tool, "a-"+tool, "res1", `{}`), contract.Revision{Schema: 1, Data: 1}) + Expect(err).To(MatchError(hostclient.ErrNoHostOps), tool) + } + }) + + It("treats an unknown tool as a bad request", func() { + rt := New(&fakeEngine{}, GinkgoT().TempDir()) + _, err := rt.Read(ctx, envelope("fine-tune-nope", "r1", "res1", "")) + Expect(err).To(MatchError(protocol.ErrBadRequest), "read") + _, err = rt.Apply(ctx, envelope("fine-tune-nope", "a1", "res1", ""), contract.Revision{Schema: 1}) + Expect(err).To(MatchError(protocol.ErrBadRequest), "apply") + _, err = rt.OpenOutput(ctx, envelope("fine-tune-status", "o1", "res1", ""), "h") + Expect(err).To(MatchError(protocol.ErrNotFound), "output") + }) + + Describe("manifest", func() { + It("declares the tools and the resource mapping", func() { + m := loadManifest() + Expect(m.ID).To(Equal("fine-tuning")) + Expect(m.Capabilities.UI.MainPage).To(Equal("jobs")) + Expect(m.Capabilities.UI.Pages).To(Equal([]string{"jobs"})) + Expect(m.Capabilities.UI.SDK.Min).To(BeEquivalentTo(2)) + Expect(m.Capabilities.UI.SDK.Max).To(BeEquivalentTo(2)) + Expect(m.Capabilities.Tools).To(Equal(Tools)) + r := m.Capabilities.Resources + Expect(r).NotTo(BeNil()) + Expect(r.Noun).To(Equal("fine-tuning job")) + Expect(r.Create).To(Equal("fine-tune-start")) + Expect(r.Inspect).To(Equal("fine-tune-status")) + kinds := map[string]string{} + for _, d := range m.Capabilities.ToolDefinitions { + kinds[d.Name] = d.Kind + } + want := map[string]string{"fine-tune-list": "read", "fine-tune-status": "read", "fine-tune-checkpoints": "read", "fine-tune-start": "create", "fine-tune-dataset": "create", "fine-tune-stop": "write", "fine-tune-export": "write"} + Expect(kinds).To(Equal(want)) + for _, d := range m.Capabilities.ToolDefinitions { + if d.Name == "fine-tune-list" { + Expect(contract.IsResourceless(d)).To(BeTrue(), "the list tool must declare no resource") + } else { + Expect(contract.IsResourceless(d)).To(BeFalse(), "%s must not be resource-less", d.Name) + } + Expect(len(d.Description)).To(BeNumerically("<", 400), d.Name) + Expect(d.Description).To(ContainSubstring("started"), "%s: the description must say that only jobs a tool started can be controlled", d.Name) + } + }) + + It("names no user or path in any tool argument", func() { + m := loadManifest() + for _, d := range m.Capabilities.ToolDefinitions { + var s struct { + Properties map[string]json.RawMessage `json:"properties"` + } + Expect(json.Unmarshal(d.InputSchema, &s)).To(Succeed()) + for name := range s.Properties { + lower := strings.ToLower(name) + Expect(contract.ForbiddenToolProperty(name) || strings.Contains(lower, "path") || strings.Contains(lower, "dir")).To(BeFalse(), + "%s: property %q names an identity or a path", d.Name, name) + } + } + }) + + It("gives the start schema the task 20 fields and the name the host needs", func() { + m := loadManifest() + found := false + for _, d := range m.Capabilities.ToolDefinitions { + if d.Name != "fine-tune-start" { + continue + } + found = true + var s struct { + Properties map[string]json.RawMessage `json:"properties"` + Required []string `json:"required"` + } + _ = json.Unmarshal(d.InputSchema, &s) + want := []string{"name", "model", "backend", "trainingType", "trainingMethod", "datasetSource", "datasetSplit", "learningRate", "numEpochs", "batchSize", "maxSteps", "adapterRank", "adapterAlpha"} + Expect(s.Properties).To(HaveLen(len(want))) + for _, w := range want { + Expect(s.Properties).To(HaveKey(w)) + } + break + } + Expect(found).To(BeTrue(), "no start definition") + }) + }) + + It("lists the caller's jobs without a resource", func() { + eng := &fakeEngine{result: map[string]any{"finetune.list": map[string]any{"jobs": []any{map[string]any{"id": "ui-job", "status": "completed"}}}}} + rt := New(eng, GinkgoT().TempDir()) + res, err := rt.Read(ctx, envelope("fine-tune-list", "r1", "", "")) + Expect(err).NotTo(HaveOccurred()) + Expect(string(res.Payload)).To(ContainSubstring("ui-job")) + Expect(eng.ops()).To(Equal([]string{"finetune.list"})) + Expect(eng.last("finetune.list").args).To(BeNil(), "the list takes no argument") + _, err = rt.Read(ctx, envelope("fine-tune-list", "r2", "", `{"userId":"other"}`)) + Expect(err).To(MatchError(protocol.ErrBadRequest), "an argument must be refused") + var nilClient *hostclient.Client + _, err = New(nilClient, GinkgoT().TempDir()).Read(ctx, envelope("fine-tune-list", "r3", "", "")) + Expect(err).To(MatchError(hostclient.ErrNoHostOps), "no endpoint") + }) + + It("sends a dataset once and returns its ref only through the resource", func() { + eng := &fakeEngine{result: map[string]any{"finetune.dataset": map[string]any{"datasetRef": "0000000a-data.jsonl"}}} + rt := New(eng, GinkgoT().TempDir()) + p, err := rt.Preview(ctx, envelope("fine-tune-dataset", "create-d1", "d1", datasetJSON)) + Expect(err).NotTo(HaveOccurred()) + Expect(p.Revision.Schema).NotTo(BeZero()) + Expect(eng.ops()).To(BeEmpty()) + Expect(string(p.Summary)).NotTo(ContainSubstring("text"), "the approval card must not carry the content") + rec, err := rt.Apply(ctx, envelope("fine-tune-dataset", "create-d1", "d1", datasetJSON), p.Revision) + Expect(err).NotTo(HaveOccurred()) + Expect(rec.State).To(Equal(contract.ReceiptCommitted)) + Expect(string(rec.Result.Payload)).To(ContainSubstring("0000000a-data.jsonl")) + _, err = rt.Apply(ctx, envelope("fine-tune-dataset", "create-d1", "d1", datasetJSON), p.Revision) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.count("finetune.dataset")).To(Equal(1)) + a := eng.last("finetune.dataset").args + Expect(a["name"]).To(Equal("data.jsonl")) + Expect(a["content"]).NotTo(BeNil()) + Expect(a).To(HaveLen(2), "host operation args") + // The resource is the dataset: its status gives the reference to the owner of the resource. + st, err := rt.Read(ctx, envelope("fine-tune-status", "r1", "d1", "")) + Expect(err).NotTo(HaveOccurred()) + Expect(string(st.Payload)).To(ContainSubstring(`"datasetRef":"0000000a-data.jsonl"`)) + Expect(string(st.Payload)).To(ContainSubstring(`"kind":"dataset"`)) + Expect(eng.ops()).To(HaveLen(1), "the status of a dataset must not call the engine") + ok, err := exists(rt, "d1") + Expect(err).NotTo(HaveOccurred()) + Expect(ok).To(BeTrue()) + // A dataset is not a job. + for _, tool := range []string{"fine-tune-checkpoints"} { + _, err := rt.Read(ctx, envelope(tool, "r2", "d1", "")) + Expect(err).To(MatchError(protocol.ErrNotFound), tool) + } + _, err = rt.Apply(ctx, envelope("fine-tune-stop", "s1", "d1", `{}`), contract.Revision{Schema: 1, Data: 1}) + Expect(err).To(MatchError(protocol.ErrNotFound), "stop") + }) + + Describe("dataset validation", func() { + big := strings.Repeat("a", 900<<10+1) + for name, args := range map[string]string{ + "empty": `{"name":"d.jsonl","content":""}`, + "too large": `{"name":"d.jsonl","content":"` + big + `"}`, + "a folder": `{"name":"a/d.jsonl","content":"x"}`, + "parent": `{"name":"../d.jsonl","content":"x"}`, + "wrong type": `{"name":"d.exe","content":"x"}`, + "only the type": `{"name":".jsonl","content":"x"}`, + "extra field": `{"name":"d.jsonl","content":"x","path":"/etc"}`, + } { + It("refuses "+name, func() { + rt := New(&fakeEngine{}, GinkgoT().TempDir()) + _, err := rt.Preview(ctx, envelope("fine-tune-dataset", "o-"+name, "d9", args)) + Expect(err).To(MatchError(protocol.ErrBadRequest)) + }) + } + + It("accepts the largest allowed dataset", func() { + rt := New(&fakeEngine{}, GinkgoT().TempDir()) + ok := `{"name":"d.jsonl","content":"` + strings.Repeat("a", 900<<10) + `"}` + _, err := rt.Preview(ctx, envelope("fine-tune-dataset", "o-ok", "d9", ok)) + Expect(err).NotTo(HaveOccurred(), "900 KiB") + }) + }) + + It("passes a dataset ref to start and keeps no job when the host refuses it", func() { + eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}}} + rt := New(eng, GinkgoT().TempDir()) + args := `{"name":"run","model":"org/base","datasetSource":"0000000a-data.jsonl"}` + _, err := rt.Apply(ctx, envelope("fine-tune-start", "create-r1", "r1", args), contract.Revision{Schema: 1}) + Expect(err).NotTo(HaveOccurred()) + Expect(eng.last("finetune.start").args["datasetSource"]).To(Equal("0000000a-data.jsonl")) + // The host refuses the reference (the per-user check is proven in the host tests); the add-on only maps the refusal and keeps nothing. + eng.result["finetune.start"] = &contract.HostError{Code: contract.HostBadRequest, Message: "datasetSource is not a dataset reference of this user"} + _, err = rt.Apply(ctx, envelope("fine-tune-start", "create-r2", "r2", `{"name":"run","model":"org/base","datasetSource":"0000000b-theirs.jsonl"}`), contract.Revision{Schema: 1}) + Expect(err).To(MatchError(protocol.ErrBadRequest)) + _, err = rt.Read(ctx, envelope("fine-tune-status", "r3", "r2", "")) + Expect(err).To(MatchError(protocol.ErrNotFound), "a refused start must not leave a job") + _, err = rt.Receipt(ctx, "create-r2") + Expect(err).To(MatchError(protocol.ErrNotFound), "a refused start must not leave a receipt") + }) +}) diff --git a/addons/fine-tuning/manifest.json b/addons/fine-tuning/manifest.json new file mode 100644 index 000000000..07d7e5532 --- /dev/null +++ b/addons/fine-tuning/manifest.json @@ -0,0 +1,285 @@ +{ + "schemaVersion": 1, + "id": "fine-tuning", + "version": "0.1.0", + "executable": "fine-tuning", + "os": "linux", + "arch": "amd64", + "host": { + "min": 1, + "max": 1 + }, + "api": { + "min": 1, + "max": 1 + }, + "protocol": { + "min": 1, + "max": 1 + }, + "dataFormat": { + "min": 1, + "max": 1 + }, + "capabilities": { + "tools": [ + "resource-exists", + "fine-tune-list", + "fine-tune-start", + "fine-tune-dataset", + "fine-tune-status", + "fine-tune-checkpoints", + "fine-tune-stop", + "fine-tune-export" + ], + "toolDefinitions": [ + { + "name": "fine-tune-list", + "kind": "read", + "description": "List the fine-tuning jobs of the person you are talking to, newest first, including jobs started in the web page. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": {}, + "additionalProperties": false + } + }, + { + "name": "fine-tune-start", + "kind": "create", + "description": "Propose a new fine-tuning job. A person confirms it in the approval card; nothing starts before that. datasetSource is a Hugging Face ID or a reference from fine-tune-dataset. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "A label for the job, 1 to 80 characters.", + "maxLength": 80 + }, + "model": { + "type": "string", + "description": "Base model as a gallery or Hugging Face ID. A local path is refused.", + "maxLength": 256 + }, + "backend": { + "type": "string", + "description": "Fine-tuning backend. The default is trl.", + "maxLength": 64 + }, + "trainingType": { + "type": "string", + "description": "Kind of training. The default is lora.", + "enum": [ + "lora", + "loha", + "lokr", + "full" + ] + }, + "trainingMethod": { + "type": "string", + "description": "Training method. The default is sft.", + "enum": [ + "sft", + "dpo", + "grpo", + "rloo", + "reward", + "kto", + "orpo" + ] + }, + "datasetSource": { + "type": "string", + "description": "Dataset as a Hugging Face ID or a reference from fine-tune-dataset. A local path is refused.", + "maxLength": 256 + }, + "datasetSplit": { + "type": "string", + "description": "Dataset split, such as train.", + "maxLength": 64 + }, + "learningRate": { + "type": "number", + "description": "Learning rate. Zero keeps the default.", + "minimum": 0 + }, + "numEpochs": { + "type": "integer", + "description": "Number of epochs. Zero keeps the default.", + "minimum": 0, + "maximum": 1000 + }, + "batchSize": { + "type": "integer", + "description": "Batch size. Zero keeps the default.", + "minimum": 0, + "maximum": 4096 + }, + "maxSteps": { + "type": "integer", + "description": "Stop after this many steps. Zero means no limit.", + "minimum": 0 + }, + "adapterRank": { + "type": "integer", + "description": "Rank of the adapter. Zero keeps the default.", + "minimum": 0, + "maximum": 1024 + }, + "adapterAlpha": { + "type": "integer", + "description": "Alpha of the adapter. Zero keeps the default.", + "minimum": 0, + "maximum": 4096 + } + }, + "required": [ + "name", + "model", + "datasetSource" + ], + "additionalProperties": false + } + }, + { + "name": "fine-tune-dataset", + "kind": "create", + "description": "Propose sending a small text dataset (up to 900 KiB, a .jsonl, .json, .csv or .txt file) to the server. A person confirms it. The reference it returns is the datasetSource of fine-tune-start, works for that person only and is lost when the server restarts. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "File name of the dataset, ending in .jsonl, .json, .csv or .txt. No folder.", + "maxLength": 255 + }, + "content": { + "type": "string", + "description": "The dataset as UTF-8 text, 1 byte to 900 KiB.", + "maxLength": 921600 + } + }, + "required": [ + "name", + "content" + ], + "additionalProperties": false + } + }, + { + "name": "fine-tune-status", + "kind": "read", + "description": "Show the status of a fine-tuning job that a tool started, or the reference of a dataset that fine-tune-dataset sent. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "resourceId": { + "type": "string", + "description": "ID of the fine-tuning job, as returned by a job start. Only a job that a tool started has one." + } + }, + "required": [ + "resourceId" + ], + "additionalProperties": false + } + }, + { + "name": "fine-tune-checkpoints", + "kind": "read", + "description": "List the saved checkpoints of a fine-tuning job that a tool started, with step and time. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "resourceId": { + "type": "string", + "description": "ID of the fine-tuning job, as returned by a job start. Only a job that a tool started has one." + } + }, + "required": [ + "resourceId" + ], + "additionalProperties": false + } + }, + { + "name": "fine-tune-stop", + "kind": "write", + "description": "Propose stopping a fine-tuning job that a tool started. A person confirms it. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "resourceId": { + "type": "string", + "description": "ID of the fine-tuning job, as returned by a job start. Only a job that a tool started has one." + }, + "saveCheckpoint": { + "type": "boolean", + "description": "Keep a checkpoint of the work so far. The default is false." + } + }, + "required": [ + "resourceId" + ], + "additionalProperties": false + } + }, + { + "name": "fine-tune-export", + "kind": "write", + "description": "Propose registering the result of a fine-tuning job that a tool started as a model. Administrators only. A model can only control jobs it started.", + "inputSchema": { + "type": "object", + "properties": { + "resourceId": { + "type": "string", + "description": "ID of the fine-tuning job, as returned by a job start. Only a job that a tool started has one." + }, + "exportFormat": { + "type": "string", + "description": "Export format. The default is the engine default.", + "enum": [ + "lora", + "merged_16bit", + "merged_4bit", + "gguf" + ] + }, + "name": { + "type": "string", + "description": "Name of the model, 1 to 64 letters, digits, dots, dashes or underscores. Leave out to let the engine choose.", + "maxLength": 64 + } + }, + "required": [ + "resourceId" + ], + "additionalProperties": false + } + } + ], + "resources": { + "noun": "fine-tuning job", + "create": "fine-tune-start", + "inspect": "fine-tune-status" + }, + "ui": { + "entry": "ui/fine-tuning.js", + "css": [ + "ui/fine-tuning.css" + ], + "sdk": { + "min": 2, + "max": 2 + }, + "pages": [ + "jobs" + ], + "navigation": [], + "mainPage": "jobs", + "icon": "ui/icon.png", + "title": "Fine-tuning", + "description": "Train an adapter on your own data, then export it as a model you can chat with." + } + } +} diff --git a/addons/fine-tuning/package/platforms.yaml b/addons/fine-tuning/package/platforms.yaml new file mode 100644 index 000000000..5dec8568b --- /dev/null +++ b/addons/fine-tuning/package/platforms.yaml @@ -0,0 +1,38 @@ +# Release platform matrix of the fine-tuning add-on package. +# +# A platform ships only while "enabled" is true. Enable a platform in the same +# change that shows its native check passing on a runner of that platform; +# until then it stays listed and disabled. The release workflow fails closed +# when no platform is enabled, and it never builds a platform that is not +# enabled here. +# +# runner: the GitHub runner label that builds and checks the platform +# natively. Required for an enabled platform. +# native_check: "executable-start": the packaged executable is extracted on +# the native runner and started. Required for an enabled platform. +# runtime: files besides the executable that the platform needs at run +# time, relative to the package root. They are taken from the +# runtime directory of the build and must exist. +schema: 1 +addon: fine-tuning +executable: fine-tuning +manifest: addons/fine-tuning/manifest.json +# The directory whose content is laid at the package root (ui/fine-tuning.js ...). +ui: addons/fine-tuning/ui/dist +platforms: + - platform: linux/amd64 + enabled: true + runner: ubuntu-24.04 + native_check: executable-start + - platform: linux/arm64 + enabled: true + runner: ubuntu-24.04-arm + native_check: executable-start + - platform: darwin/arm64 + enabled: true + runner: macos-14 + native_check: executable-start + - platform: darwin/amd64 + enabled: true + runner: macos-15-intel + native_check: executable-start diff --git a/addons/fine-tuning/ui/.gitignore b/addons/fine-tuning/ui/.gitignore new file mode 100644 index 000000000..1225aa2fa --- /dev/null +++ b/addons/fine-tuning/ui/.gitignore @@ -0,0 +1,2 @@ +# Written by build.mjs; the package build makes it. +/dist/ diff --git a/addons/fine-tuning/ui/build.mjs b/addons/fine-tuning/ui/build.mjs new file mode 100644 index 000000000..1c980473c --- /dev/null +++ b/addons/fine-tuning/ui/build.mjs @@ -0,0 +1,13 @@ +import { fileURLToPath } from 'node:url' +import { dirname } from 'node:path' +import { buildAddonUI } from '../../internal/uibuild/uibuild.mjs' + +const here = dirname(fileURLToPath(import.meta.url)) + +export async function build() { + return buildAddonUI({ root: here, name: 'fine-tuning', rgb: [92, 74, 150] }) +} + +if (process.argv[1] === fileURLToPath(import.meta.url)) { + build().then((dir) => console.log(`built ${dir}`), (e) => { console.error(e); process.exit(1) }) +} diff --git a/addons/fine-tuning/ui/icon.png b/addons/fine-tuning/ui/icon.png new file mode 100644 index 0000000000000000000000000000000000000000..5f76999d12e10323c55c374a0e3ceddbef475fff GIT binary patch literal 5362 zcmcgw`8Sl|-+so35*ali#E=%dNXRxAGDOIlEM+H4wit%7ltK3F`@R*4L3W|Zo?WtJ z-}ikC^G@IMzJJ2^{pq>S^|{Y|o^#)y>$=YM3DMMeOh>~)0{{RWQU#$603h-b1YD#d ze_ps0S^)qAKqBOIJX6;vFS+UHrc-TiHFhdc_;Y^^g&VM34CTtq45!%)<)TCs4BLu9 z9O{OJ@#PyyvaX49N$Ik#Is*fOJ=r$ZlO7M`U%=mhuHXL2EA_Kch{7L!STA)XOBa(H zGZa&=Uvzrb%cHI9rCX*lOe#6rY}jgKMttE(+9QnVEdcP`%OP$TbBpR?^FBJfgCJK? z-Vm=X?}H?M?Tn6Rt0+!x52U|(yn};I#-#e7)pR)>gDOlx+QrXtNy2p9*|uY5Rf#U4 z*>csI3;X;LPahn6UHeth1*ndmuCL75aL8i@kTf0UF?)x@HKWZsHRH700apnuZB`XC z4R;&ubVsLnL4#O1CHoFH@bzxrP42uiR@`A7C4v=lLoGcB>i5j(Vug!Ugb_I58w(I= zxe)nes%TU3Q;X>!kxDuf~>D?}bPse9few(VcYP_j+>FyF0g(4(Ew? zbxf^6VHt(CE6D@6JiGO2x6fH24?3>N>^S@4J9&w^#tjUJp$Wzx$IjEDLe^~ZiQlgP zlfS4r<@BBoUq{T-@GQvIb-o?k*T*{|n7@x-NQ@CXQr0D^84?kjR~26T4J#rLcDn0J z7~V&LVcsrN=qiUkeWARY*q`9yL;mvZa^wj;48gfUA)Ov~6t$Y+a{~e8+3ZP5+Mx=7 zx&g~tXuG-@q*faNCFW3J-)Uz&B$2mT8R~$@J{3H2op4BL&eKiqe)|*gAg%ib|7w`-#xcgn)|eGoG0zorsS%HfaDV zUA{C3s5}2L+H-ghdMb>7&lc^>jB;8Zuc`2^8+#g&F_0N@f{rWH zcecP7ZOsIaBuYiIxvEZLsTq{SWL%@tqN1eHgwK|}jPRFrV|?V6u=@3kaH85-`fJp0 z;=-550a)5y#U7ILq6w#R0JeL_yyx;J=~_}j?ak&#Bfa8@?N2Qjt?fuLF>2;M6ld~? zAN@Edd6z0vu3JUp)!a1~;1+kH^Swba`Q#xY_u=Z1;5lsYR;Phn(!b%u<0wcBYSMb{Kp^+M?v*fDpL=CR~~7TfRnw4 zuasKYRTkk0$nV&_6Ftp9LWvWx{2F!kvjpx3?-Z#+w>37Ozy9q>IWIs340nB{+^lhh zl{S`&C%4WS91Z{gmgE2l} z0`CYf&T&tXupl{FR51F9HALVuYJV8s{T2y=iygn>w~+C$)TBets3O(fMv6w&H)l;# z?P{C`G~J%6`TyB}w1sxQ_tTgbcx3v4W}f+eF_Qw$@j#wU+W9K=QHINVg2%?x z72{r~VE+UU`FqMA*53sPz4cG#0K?#uDDGRbvX4kFkPdVGdG_DhH6o^x?=5+3_c z!cRv_W&aYT%M|S=_WnZl*U{_&{Wv437scb?P>ncFsHJ7xSX~`tNFsvrH9T0uLHOKM zP`(LtQ(kOE%V%%uIn%GNs0Uf>`*RP`+gj%-_HXvLZKM2NgA4JN`fhkXX*4)&IU!%^^nmMg_W)kB^3&?3}UlX25m(Q#%E`DAAa4seD$JUyqX~s$To`KNv zk;p+kLX&)1aw0=kelKp}zopo1l`0gJ3mH%!rhRXexjS>=%p^jqj#Gz;6?C@zrwRPC z$z`~iao4;{(eHH);n0b`ILrvZE$955&91~XA%jF!@U4rBL2VM14x=_CnqRH=f3yao_q|aBH=D`O{tqC{F!rC{9digc2i8bUuM;|-c}<3;Tl;~azj$j7 zhYKeO+0O?L+A>>0!*Qv2l;zB6ioE-MfOx#PCbRmLRVN|+&U*eI!Yu>wV*D~5m1Wi; z_{pT4Lx%~R$xceF==}CEUI$}WJ2T>YYzUFLy#kh+As7)_90(2GdpO3>CRMMx&t;qPWcT8_R1e4i03t2Bp zxnJ2Lg47r{fNy~g!2I!k>*ro^Cy9N|_8G>eDJRm$2I10|KvYg;?$r$bF^Vz)qS@t6 z)dp6P*DO>(8typOVGK7XQJ>AaDNAx={V7AHg<9mtl(X;Jsr>x@e>ASgzgpbSWBa@$ zLME+qZ6TW|sU8}4UDr9xWc0&|>0I}UOH>^*po2e`xx3S;)}?7OLgdUGy?@$n>0Zmw zag!Fr7DIv?-fB7t^T_kH5MxR`zuTnP@`!F z$xs|0y;;Yrh}|%5T7U-mqMrV2Kq}klIn7lgJm>xUjF(okuQTaO1HOiT|%e&$qHiKi|-xDi2a!ki*wKQ%exCqL?X!hnc1 zlmAbSLkiE_X-tIYZd0j!{A}4sjx}$3DyqTm$@)buKW2b^dF0lOSYMe5+uu%(wM%BZ zPVc4GxQoL{G(LgoD_!QU8tMD^*pzH$pZvj>fu6r$_0Z-ZX8>)_Xm9 zBv@Y~8jxq&MeQn%!7BQ*GF&sy^6rBcJe84^t*n27RBv98+hQ0p=jDCf?QJj6_`JTN zW#ON)-+z0Bw)TzF@0dv~sTVb$K3%PQ%Z^V^yQ^tv8@P%Wz);aXQG7jvsJC6kJPRuQ4DKDO>kSK~O{ZlnWnkm%e8n>g!g zNr#eZ#i+r&=XR*y?j`gXyMgdDg3Qp??Grq(onxJ$LnJ1rZ+Ct_62E0YrZ&y))24)m z(<CjursdwwoGpyOhB${HuZaRU;D&qkZ3R1xZ;-5ZP13e0tbXk6&khMf1 zfs&}iCue#;58a#6R;R^0kBOOmX*h+}gf`;$~DA$-6Pkw|0xx8P#ce{OI zTU8=F;u718zK~2X*OF3U7+I2c^Zcuu#c_M%)yjq1f9?hl&Gj??S&S^oDRwY;{esNn z=Jvq8$6u)EHaZ0`sO6(w#e;P1f9|qh`NxTLX5?u}oJ@c{6x^Pp+P~QT=B?MLE5pjZ zx8aXNh9;lZ;0dwoI{22ef-hDlzHaLiR7>s`(G}8A0wzsw^{egK3?_cGy=;k;lU(Ka zC?IJtVYmJg-QRKmH1N#s9f}{r9vy$OHghqaee&o?l5hCVCz6xUO(^Q(wo-zdym0QS zsLw)s04k$8ZxBzhK6#U9Kyz)!00X=`^&hV*flLkru$l^Q5A5w24f{1%^j}F8y#N4%94fh>tX@p{C`IoTeJ7LiuR{rWHeUy|mrd+=-F1 z*)$TOOWyUGOy)1>wIk=0&8tSjk%2D@?@(d7_O-lgFK=38c@MrxlhFI1Z)!r7Jx(U% zuUmAk=SQw=vIaqm`J+2T5}E;rK8LbXVII3>oV)rN1ck39m&uusrT1~A#yN6kP6D8h zOVE{6)IFHbsV!?N(zuaNP1lgo4gVUPVM#9mnzMmP%1d(pexvIEjplW zq+I362Ukb?_V$HdTJd&k-qa5GfqS;5GZQRe<@?Cl5u)c^!~f~8dgAg00H93yHw*ZG z5$En0_zmn&iP}Q}kRc71NF*ZbcAgp88dUsF)*fOMAklBQgJ=NG8vx>tHqPl1&lP{{ zhaCIq7>bnt3Yb{LoW*lw`U8Amn1+o&0B7Ko*VZG_g;SQ3)2 z0fSagrAuG^Rf9x>+7g5s-2PlbZ={|8^K4|mxA>p}1bH^TC~(P(1p#fym43!Q>r&#B zPKU8RM-ATKRAR^83k|8Z?yhO5A?SVPX=!7FXhBO1=fo&r)+06TtN0~Yw$1)Yq?oT> z8dBbY)sK=v>82k}fGChC!{a0w#7x5#%2?uJWkp+rE||7$ao|k9z@S8LB)zfa*aV6i zcYjg3Drf-umoVEp=N-+b500doK*H|Dnst{)TD3cQ$)PsS-oQXaZM3p!deqx0bICf~ zh$rY+6kFInAeJg>PAn$)PX8@x=``o_EA#cC=dyb5D2dn;M^R(<-_{CxIZ@V5ty1gor(d+R%}(!({1CXnE{9AdiuDW7f`%{V()!L z^6QySjIJh;SdAVu4+dT~G`zF0C|%i~M?LtUO+%ss{dV8HWVu#Ug!$~SgT1zOZN zDpqVS_KTX54H&Qd{q}W9%&teA$aPpYHp^anG{3apjGajbuo<)t7t*AoHvC`w&cCsO zjie~P&tKqfNl6@xt8Y2lx2i2rL;XFh*#sbc72Q)-Q=2ck8$!+SO%=Brhh#O+$Vl7a zRDkLMHos#RBXoo;OHuh2kZ84eq%P6?{DC)->7t@a9k4V2eiyWFoL^cBX7~nogvYV5 zIqulMbOPzrk&BSz{GMLVp5m{9I{L2ih?lWfor>bm_wxmPKX+6Ey$1$* zQ};OApWtGqVA+|4PG>z|b~K-@uZL9#HMfIe6ui{R3u-Xecj9sKvBf7}4 z2o|-g!5Ku4c<&NX54j2rJr1^f*@4Z-nX}F&Vy3{p#uy( zyzOejaWJY@E#QsOoP@0o=aa{M!@k-^xV6mIEE#P5Q3K%#V^QBck9&jWb)q287bMz4 y+|Z-@hZ%jp`ydGn03?!*CN_&axup6hQ($1C(I&WS4CzHi0Hl%zqEx{w;C}#|j_F7M literal 0 HcmV?d00001 diff --git a/addons/fine-tuning/ui/package.json b/addons/fine-tuning/ui/package.json new file mode 100644 index 000000000..45d0c7249 --- /dev/null +++ b/addons/fine-tuning/ui/package.json @@ -0,0 +1,8 @@ +{ + "name": "localai-fine-tuning-ui", + "private": true, + "version": "0.1.0", + "description": "Native UI of the fine-tuning add-on: a standalone ES module that the host loads after its own build.", + "type": "module", + "scripts": { "build": "node build.mjs", "test": "node test/run.mjs" } +} diff --git a/addons/fine-tuning/ui/src/FineTune.jsx b/addons/fine-tuning/ui/src/FineTune.jsx new file mode 100644 index 000000000..8879aa979 --- /dev/null +++ b/addons/fine-tuning/ui/src/FineTune.jsx @@ -0,0 +1,1350 @@ +/* eslint-disable no-unused-vars -- components used only inside JSX look unused to this config */ +// The fine-tuning page, ported from core/http/react-ui/src/pages/FineTune.jsx +// with the port table: React, the translator, the shared components, the +// session-bound fetch and the viewer come from ctx; the job views are the +// shared ones of addons/internal/tools-ui. Class names, test ids and keys are +// the core page's. +import { percentOf, ACTIVE } from './lib/progress.js' +import { createLossChart } from './LossChart.jsx' + +const TRAINING_METHODS = ['sft', 'dpo', 'grpo', 'rloo', 'reward', 'kto', 'orpo'] +const ADAPTER_KINDS = ['lora', 'loha', 'lokr'] +const FALLBACK_BACKEND = 'trl' +const OPTIMIZERS = ['adamw_torch', 'adamw_8bit', 'sgd', 'adafactor', 'prodigy'] +const MIXED_PRECISION_OPTS = ['', 'fp16', 'bf16', 'no'] +const QUANT_PRESETS = ['q4_k_m', 'q5_k_m', 'q8_0', 'f16', 'q4_0', 'q5_0'] + +const BUILTIN_REWARDS = [ + { name: 'format_reward', params: [] }, + { name: 'reasoning_accuracy_reward', params: [] }, + { name: 'length_reward', params: [{ key: 'target_length', default: '200' }] }, + { name: 'xml_tag_reward', params: [] }, + { name: 'no_repetition_reward', params: [] }, + { name: 'code_execution_reward', params: [] }, +] + +const BADGE = { + queued: '', loading_model: 'dk-badge--warn', loading_dataset: 'dk-badge--warn', training: 'dk-badge--accent', + saving: 'dk-badge--accent', completed: 'dk-badge--ok', failed: 'dk-badge--error', stopped: '', +} +const EXTRA_HANDLED = ['max_seq_length', 'save_total_limit', 'hf_token', 'eval_strategy', 'eval_steps', 'eval_split', 'eval_dataset_source', 'eval_split_ratio', 'voice', 'val_dataset'] + +function etaText(t, seconds) { + if (!seconds || seconds <= 0) return '' + const h = Math.floor(seconds / 3600) + const m = Math.floor((seconds % 3600) / 60) + if (h > 0) return t('run.eta', { time: `${h}h ${m}m` }) + if (m > 0) return t('run.eta', { time: `${m} min` }) + return t('run.etaSeconds', { seconds: Math.max(1, Math.floor(seconds)) }) +} + +const kindOf = (job) => `${job.training_type || 'lora'}, ${job.training_method || 'sft'}` +const startedAt = (job) => { + const d = new Date(job.created_at) + return Number.isNaN(d.getTime()) ? (job.created_at || '') : d.toLocaleString([], { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' }) +} + +// `shared` is the shared job views: { tools, readSse, ToolSteps, Checks, StageLine, +// JobLog, ToolOff }. Returns the page and, for tests, the parts it is made of. +export function createFineTune(ctx, api, shared) { + const { h, Fragment, t, Icon, ui, viewer } = ctx + const { useState, useEffect, useRef, useCallback, useMemo } = ctx.React + const { FT_STAGES, TERMINAL, appendLog, blocked, fineTuneChecks, logLines, looksLikeMemoryFailure, machineFacts, warned } = shared.tools + const { ToolSteps, Checks, StageLine, JobLog, ToolOff, readSse } = shared + const { LossChart } = createLossChart(ctx) + + function StatusBadge({ status }) { + return {ctx.tOr(`fineTune.status.${status}`, String(status || '').replace(/_/g, ' '))} + } + + function Field({ id, label, hint, className = '', children }) { + return ( +
+ + {children} + {hint &&

{hint}

} +
+ ) + } + + function Select({ id, value, onChange, children }) { + return ( + + + + ) + } + + function Section({ done, title, children }) { + return ( +
+

+ + {title} +

+
{children}
+
+ ) + } + + function KeyValueEditor({ entries, onChange }) { + const update = (i, field, val) => onChange(entries.map((e, idx) => (idx === i ? { ...e, [field]: val } : e))) + return ( +
+ {entries.map((entry, i) => ( +
+ update(i, 'key', e.target.value)} placeholder={t('form.key')} aria-label={`Extra option ${i + 1} key`} /> + update(i, 'value', e.target.value)} placeholder={t('form.value')} aria-label={`Extra option ${i + 1} value`} /> + +
+ ))} + +
+ ) + } + + function CopyButton({ text }) { + const [copied, setCopied] = useState(false) + const copy = (e) => { + e.stopPropagation() + navigator.clipboard?.writeText(text).then(() => { setCopied(true); setTimeout(() => setCopied(false), 1500) }).catch(() => {}) + } + return ( + + ) + } + + // The foot of the stop layer. The host draws the dialog around it. + function StopDialogBody({ onChoose, close }) { + return ( +
+ + + +
+ ) + } + + // The foot of the delete layer. + function DeleteDialogBody({ onConfirm, close }) { + return ( +
+ + +
+ ) + } + + // A running, failed or finished job: progress, the loss chart, the log, the + // checkpoints and what to do with the result. The server streams progress + // events while the job is active; a finished job is read from the job record. + // `seed` starts the view with an event and checkpoints already read; tests use + // it to draw a state the stream would reach. + function JobView({ job, onStop, onReuse, onResume, onTerminal, exportCheckpoint, seed }) { + const isAdmin = viewer.isAdmin() + const [events, setEvents] = useState([]) + const [latest, setLatest] = useState(seed?.latest ?? null) + const [log, setLog] = useState([]) + const [connecting, setConnecting] = useState(!seed) + const [checkpoints, setCheckpoints] = useState(seed?.checkpoints || []) + const previousRef = useRef(null) + const terminalRef = useRef(onTerminal) + terminalRef.current = onTerminal + + useEffect(() => { + previousRef.current = null + setEvents([]) + setLatest(null) + setLog([]) + if (!job || !ACTIVE.includes(job.status)) { setConnecting(false); return undefined } + setConnecting(true) + // The progress stream is a fetch read with readSse: it carries the + // session and the base path of sdk.core.fetch. Unmounting aborts it. + const controller = new AbortController() + const onEvent = (data) => { + setConnecting(false) + setLatest(data) + setLog(prev => appendLog(prev, logLines(data, previousRef.current))) + previousRef.current = data + if (data.loss > 0 || data.eval_loss > 0 || data.learning_rate > 0 || data.grad_norm > 0) { + setEvents(prev => { + const last = prev[prev.length - 1] + return last && last.current_step === data.current_step ? [...prev.slice(0, -1), data] : [...prev, data] + }) + } + if (TERMINAL.includes(data.status)) { controller.abort(); terminalRef.current?.() } + } + ;(async () => { + try { + const response = await api.fineTune.progress(job.id, controller.signal) + if (response.ok) await readSse(response, onEvent, controller.signal) + } catch (_) { /* the stream ended; the job list poll shows the state */ } + if (!controller.signal.aborted) setConnecting(false) + })() + return () => controller.abort() + // eslint-disable-next-line react-hooks/exhaustive-deps -- keyed on the job id: the effect restarts per job, not per poll refresh + }, [job?.id]) + + const status = latest?.status || job.status + const active = ACTIVE.includes(status) + const failed = status === 'failed' + // A running job can read 0% before it has reported anything. That 0 is + // unknown, not a measurement, so the page draws no figure for it. + const progress = percentOf(latest ? { ...latest, status } : status === 'completed' ? { status } : null) + const known = progress != null + + useEffect(() => { + if (!job || active) return + api.fineTune.listCheckpoints(job.id).then(r => setCheckpoints(r.checkpoints || [])).catch(() => {}) + // eslint-disable-next-line react-hooks/exhaustive-deps -- keyed on the job id: the effect restarts per job, not per poll refresh + }, [job?.id, active]) + + const message = latest?.message || job.message || '' + const lastCheckpoint = checkpoints.length ? checkpoints[checkpoints.length - 1] : null + const [fixes, setFixes] = useState({ batch: true, checkpointing: true }) + + const stageLabels = Object.fromEntries(FT_STAGES.map(s => [s, t(`fineTune.status.${s}`)])) + const openStop = () => ctx.openDialog({ + title: t('stop.title'), + description: t('stop.text'), + render: ({ close }) => onStop(job.id, keep)} />, + }) + + return ( +
+ {failed && ( +
+ +
+

+ {latest?.current_step > 0 + ? t('fineTune.failedAt', { step: latest.current_step, total: latest.total_steps }) + : t('fineTune.failed')} +

+

{message || t('fineTune.noMessage')}

+
+
+ )} + +
+
+
+

{job.model}

+

+ {t('fineTune.jobMeta', { backend: job.backend || FALLBACK_BACKEND, method: job.training_method || 'sft', kind: job.training_type || 'lora', started: startedAt(job) })} + · {job.id?.slice(0, 8)} +

+
+
+ + {active && ( + + )} +
+
+ + {connecting && !latest && active ? ( +

{t('run.connecting')}

+ ) : ( + <> + {(latest || status === 'completed') && ( +
+ {known + ? {progress.toFixed(0)}% + : {t('run.unknownPercent')}} + {latest && latest.total_steps > 0 && {t('run.step')} {latest.current_step} {t('run.of')} {latest.total_steps}} + {latest && latest.total_epochs > 0 && {t('run.epoch')} {latest.current_epoch?.toFixed(1)} {t('run.of')} {latest.total_epochs?.toFixed(0)}} + {active && etaText(t, latest?.eta_seconds) && {etaText(t, latest.eta_seconds)}} + {latest?.extra_metrics?.tokens_per_second > 0 && {latest.extra_metrics.tokens_per_second.toFixed(0)} {t('run.tokens')}} +
+ )} + {(active || latest) && ( +
+ +
+ )} + {active && ( + + )} + {!failed && message &&

{message}

} + {status === 'completed' && ( +

+ {events.length && events[events.length - 1].loss > 0 + ? t('fineTune.finishedLoss', { loss: events[events.length - 1].loss.toFixed(4) }) + : t('fineTune.finished')} +

+ )} + {status === 'stopped' &&

{t('fineTune.stopped')}

} + + )} +
+ + {failed && ( +
+

{t('fineTune.tryTitle')}

+ {looksLikeMemoryFailure(message) ? ( + <> +

{t('fineTune.tryText')}

+ + + + ) : ( +

{t('fineTune.tryPlain')}

+ )} +
+ + {lastCheckpoint && ( + + )} +
+
+ )} + + {(active || events.length > 0) && } + {(active || log.length > 0) && } + + {!active && checkpoints.length > 0 && ( +
+

{t('checkpoints.title')}

+
+ + + + + {checkpoints.map(cp => ( + + + + + + + + ))} + +
{t('checkpoints.title')}
{t('checkpoints.step')}{t('checkpoints.epoch')}{t('checkpoints.loss')}{t('checkpoints.path')}{t('checkpoints.actions')}
{cp.step}{cp.created_at}{cp.epoch?.toFixed(2)}{cp.loss?.toFixed(4)}{cp.path} +
+ + {isAdmin && ( + + )} +
+
+
+
+ )} +
+ ) + } + + // What to do with a finished job: export it as a model, then use it. + function ExportForm({ job, prefilledCheckpoint, seed }) { + const [checkpoints, setCheckpoints] = useState(seed?.checkpoints || []) + const [exportFormat, setExportFormat] = useState('lora') + const [quantMethod, setQuantMethod] = useState('q4_k_m') + const [modelName, setModelName] = useState('') + const [selectedCheckpoint, setSelectedCheckpoint] = useState('') + const [exporting, setExporting] = useState(false) + const [message, setMessage] = useState('') + const [exportedModelName, setExportedModelName] = useState(job?.export_status === 'completed' ? job.export_model_name || '' : '') + const [exportFailed, setExportFailed] = useState(false) + const pollRef = useRef(null) + + useEffect(() => { + if (!job) return + api.fineTune.listCheckpoints(job.id).then(r => setCheckpoints(r.checkpoints || [])).catch(() => {}) + // eslint-disable-next-line react-hooks/exhaustive-deps -- keyed on the job id: the effect restarts per job, not per poll refresh + }, [job?.id]) + + useEffect(() => { if (prefilledCheckpoint) setSelectedCheckpoint(prefilledCheckpoint.path || '') }, [prefilledCheckpoint]) + + // Sync export state from the job record (first load, list refresh). + useEffect(() => { + if (!job) return + if (job.export_status === 'exporting') { + setExporting(true); setExportFailed(false) + setMessage(job.export_message || t('export.working')) + } else if (job.export_status === 'completed' && job.export_model_name) { + setExporting(false); setExportFailed(false) + setExportedModelName(job.export_model_name) + setMessage(t('export.done', { name: job.export_model_name })) + } else if (job.export_status === 'failed') { + setExporting(false); setExportFailed(true) + setMessage(t('export.failed', { message: job.export_message || t('export.unknown') })) + } + // eslint-disable-next-line react-hooks/exhaustive-deps -- keyed on the job id: the effect restarts per job, not per poll refresh + }, [job?.export_status, job?.export_model_name, job?.export_message]) + + useEffect(() => { + if (!exporting || !job) return undefined + pollRef.current = setInterval(async () => { + try { + const updated = await api.fineTune.getJob(job.id) + if (updated.export_status === 'completed') { + setExporting(false); setExportFailed(false) + const name = updated.export_model_name || modelName || 'exported model' + setExportedModelName(name) + setMessage(t('export.done', { name })) + clearInterval(pollRef.current) + } else if (updated.export_status === 'failed') { + setExporting(false); setExportFailed(true) + setMessage(t('export.failed', { message: updated.export_message || t('export.unknown') })) + clearInterval(pollRef.current) + } else if (updated.export_status === 'exporting' && updated.export_message) { + setMessage(updated.export_message) + } + } catch (_) { /* try again at the next tick */ } + }, 3000) + return () => clearInterval(pollRef.current) + // eslint-disable-next-line react-hooks/exhaustive-deps -- keyed on the job id: the effect restarts per job, not per poll refresh + }, [exporting, job?.id]) + + const handleExport = async () => { + setExporting(true); setExportFailed(false) + setMessage(t('export.working')) + setExportedModelName('') + try { + await api.fineTune.exportModel(job.id, { + name: modelName || undefined, + checkpoint_path: selectedCheckpoint || job.output_dir, + export_format: exportFormat, + quantization_method: exportFormat === 'gguf' ? quantMethod : '', + model: job.model, + }) + } catch (e) { + setMessage(t('export.failed', { message: e.message })) + setExportFailed(true); setExporting(false) + } + } + + if (!job || !TERMINAL.includes(job.status)) return null + + return ( + <> + {exportedModelName && !exportFailed && ( +
+ +
+

{t('export.nextTitle', { name: exportedModelName })}

+

{t('export.nextText')}

+ +
+
+ )} +
+

{t('export.title')}

+

{t('export.lede')}

+
+ {checkpoints.length > 0 && ( + + + + )} +
+ + + + {exportFormat === 'gguf' && ( + + setQuantMethod(e.target.value)} placeholder="q4_k_m, bf16, f32" /> + {QUANT_PRESETS.map(q => + + )} +
+ + setModelName(e.target.value)} placeholder="my-finetuned-model" /> + +
+ +
+ {message && (!exportedModelName || exportFailed) && ( +

+ {exporting && } {message} +

+ )} +
+
+ + ) + } + + // The wrapper of the export controls. Registering a model is for + // administrators; anyone else sees why the controls are missing. + function ExportPanel({ job, prefilledCheckpoint, seed }) { + if (!job || !TERMINAL.includes(job.status)) return null + if (!viewer.isAdmin()) { + return ( +
+

{t('export.adminOnly.title')}

+

{t('export.adminOnly.text')}

+
+ ) + } + return + } + + // The machine facts for the checks: the one request useResources makes, as + // a poll that waits while the tab is hidden and catches up on return. + function useResources(interval) { + const [resources, setResources] = useState(null) + useEffect(() => { + let off = false + const read = async () => { + if (typeof document !== 'undefined' && document.hidden) return + try { + const data = await api.resources.get() + if (!off) setResources(data) + } catch (_) { /* the next tick tries again */ } + } + read() + const timer = setInterval(read, interval) + document.addEventListener('visibilitychange', read) + return () => { off = true; clearInterval(timer); document.removeEventListener('visibilitychange', read) } + }, [interval]) + return resources + } + + function FineTuneBody({ path }) { + const isAdmin = viewer.isAdmin() + const resources = useResources(10000) + const facts = useMemo(() => machineFacts(resources), [resources]) + + const [jobs, setJobs] = useState([]) + const [jobsLoaded, setJobsLoaded] = useState(false) + const [selectedJob, setSelectedJob] = useState(null) + const [loading, setLoading] = useState(false) + const [error, setError] = useState('') + const [backends, setBackends] = useState(null) + const [exportCheckpoint, setExportCheckpoint] = useState(null) + const initialConfigRef = useRef(null) + + // Form state + // A deep link names the base model: ?model= or the first path segment. + const [model, setModel] = useState(() => ctx.modelFrom(path) || '') + const [backend, setBackend] = useState(FALLBACK_BACKEND) + const [trainingMethod, setTrainingMethod] = useState('sft') + const [trainingType, setTrainingType] = useState('lora') + const [datasetSource, setDatasetSource] = useState('') + const [datasetFile, setDatasetFile] = useState(null) + const [datasetSplit, setDatasetSplit] = useState('') + const [numEpochs, setNumEpochs] = useState(3) + const [batchSize, setBatchSize] = useState(2) + const [learningRate, setLearningRate] = useState(0.0002) + const [learningRateText, setLearningRateText] = useState('0.0002') + const [adapterRank, setAdapterRank] = useState(16) + const [adapterAlpha, setAdapterAlpha] = useState(16) + const [adapterDropout, setAdapterDropout] = useState(0) + const [targetModules, setTargetModules] = useState('') + const [gradAccum, setGradAccum] = useState(4) + const [warmupSteps, setWarmupSteps] = useState(5) + const [maxSteps, setMaxSteps] = useState(0) + const [saveSteps, setSaveSteps] = useState(500) + const [weightDecay, setWeightDecay] = useState(0) + const [maxSeqLength, setMaxSeqLength] = useState(2048) + const [optimizer, setOptimizer] = useState('adamw_torch') + const [gradCheckpointing, setGradCheckpointing] = useState(false) + const [seed, setSeed] = useState(0) + const [mixedPrecision, setMixedPrecision] = useState('') + const [extraOptions, setExtraOptions] = useState([]) + const [liquidAudioVoice, setLiquidAudioVoice] = useState('') + const [liquidAudioValDataset, setLiquidAudioValDataset] = useState('') + const [hfToken, setHfToken] = useState('') + const [showAdvanced, setShowAdvanced] = useState(false) + const [resumeFromCheckpoint, setResumeFromCheckpoint] = useState('') + const [saveTotalLimit, setSaveTotalLimit] = useState(0) + const [evalEnabled, setEvalEnabled] = useState(false) + const [evalStrategy, setEvalStrategy] = useState('steps') + const [evalSteps, setEvalSteps] = useState(0) + const [evalSplit, setEvalSplit] = useState('') + const [evalDatasetSource, setEvalDatasetSource] = useState('') + const [evalSplitRatio, setEvalSplitRatio] = useState(0.1) + const [rewardFunctions, setRewardFunctions] = useState([]) + const [showAddCustomReward, setShowAddCustomReward] = useState(false) + const [customRewardName, setCustomRewardName] = useState('') + const [customRewardCode, setCustomRewardCode] = useState('') + + const loadJobs = useCallback(async () => { + try { + const data = await api.fineTune.listJobs() + const list = data || [] + setJobs(list) + setSelectedJob(prev => { + if (!prev) return prev + const fresh = list.find(j => j.id === prev.id) + return fresh ? { ...prev, ...fresh } : prev + }) + } catch (_) { /* the next poll tries again */ } finally { + setJobsLoaded(true) + } + }, []) + + useEffect(() => { + loadJobs() + const interval = setInterval(loadJobs, 10000) + return () => clearInterval(interval) + }, [loadJobs]) + + useEffect(() => { + api.fineTune.listBackends() + .then(data => { + const list = Array.isArray(data) ? data : [] + setBackends(list) + if (list.length > 0) setBackend(prev => (list.some(b => b.name === prev) ? prev : list[0].name)) + }) + .catch(() => setBackends(null)) + }, []) + + const backendNames = backends && backends.length > 0 ? backends.map(b => b.name) : [FALLBACK_BACKEND] + const isAdapter = ADAPTER_KINDS.includes(trainingType) + + const handleSubmit = async (e) => { + e.preventDefault() + setLoading(true) + setError('') + try { + let dsSource = datasetSource + if (datasetFile) { + const result = await api.fineTune.uploadDataset(datasetFile) + dsSource = result.path + } + const extra = {} + if (maxSeqLength) extra.max_seq_length = String(maxSeqLength) + if (hfToken.trim()) extra.hf_token = hfToken.trim() + if (saveTotalLimit > 0) extra.save_total_limit = String(saveTotalLimit) + if (evalEnabled) { + extra.eval_strategy = evalStrategy || 'steps' + if (evalSteps > 0) extra.eval_steps = String(evalSteps) + if (evalSplit.trim()) extra.eval_split = evalSplit.trim() + if (evalDatasetSource.trim()) extra.eval_dataset_source = evalDatasetSource.trim() + if (evalSplitRatio > 0 && evalSplitRatio !== 0.1) extra.eval_split_ratio = String(evalSplitRatio) + } else { + extra.eval_strategy = 'no' + } + for (const { key, value } of extraOptions) if (key.trim()) extra[key.trim()] = value + // The Python backend reads `voice` and `val_dataset` from extra_options. + if (backend === 'liquid-audio') { + if (liquidAudioVoice) extra.voice = liquidAudioVoice + if (liquidAudioValDataset.trim()) extra.val_dataset = liquidAudioValDataset.trim() + } + + const req = { + model, + backend, + training_method: trainingMethod, + training_type: trainingType, + dataset_source: dsSource, + dataset_split: datasetSplit || undefined, + num_epochs: numEpochs, + batch_size: batchSize, + learning_rate: learningRate, + adapter_rank: isAdapter ? adapterRank : 0, + adapter_alpha: isAdapter ? adapterAlpha : 0, + adapter_dropout: isAdapter && adapterDropout > 0 ? adapterDropout : undefined, + target_modules: isAdapter && targetModules.trim() ? targetModules.split(',').map(s => s.trim()) : undefined, + gradient_accumulation_steps: gradAccum, + warmup_steps: warmupSteps, + max_steps: maxSteps > 0 ? maxSteps : undefined, + save_steps: saveSteps > 0 ? saveSteps : undefined, + weight_decay: weightDecay > 0 ? weightDecay : undefined, + gradient_checkpointing: gradCheckpointing, + optimizer, + seed: seed > 0 ? seed : undefined, + mixed_precision: mixedPrecision || undefined, + resume_from_checkpoint: resumeFromCheckpoint || undefined, + extra_options: Object.keys(extra).length > 0 ? extra : undefined, + reward_functions: trainingMethod === 'grpo' && rewardFunctions.length > 0 ? rewardFunctions : undefined, + } + + const resp = await api.fineTune.startJob(req) + setResumeFromCheckpoint('') + // Job submitted: rebaseline so leaving the page no longer warns. + initialConfigRef.current = JSON.stringify(getFormConfig()) + await loadJobs() + setSelectedJob({ ...req, id: resp.id, status: 'queued', created_at: new Date().toISOString() }) + } catch (err) { + setError(err.message) + } + setLoading(false) + } + + const handleStop = async (jobId, saveCheckpoint = true) => { + try { + await api.fineTune.stopJob(jobId, saveCheckpoint) + await loadJobs() + } catch (err) { + setError(err.message) + } + } + + const confirmDelete = async (jobId) => { + try { + await api.fineTune.deleteJob(jobId) + if (selectedJob?.id === jobId) setSelectedJob(null) + await loadJobs() + } catch (err) { + setError(err.message) + } + } + + const askDelete = (jobId) => ctx.openDialog({ + title: t('jobs.deleteDialogTitle'), + description: t('jobs.deleteDialogText'), + render: ({ close }) => confirmDelete(jobId)} />, + }) + + const getFormConfig = () => { + const extra = {} + for (const { key, value } of extraOptions) if (key.trim()) extra[key.trim()] = value + if (backend === 'liquid-audio') { + if (liquidAudioVoice) extra.voice = liquidAudioVoice + if (liquidAudioValDataset.trim()) extra.val_dataset = liquidAudioValDataset.trim() + } + return { + model, backend, + training_method: trainingMethod, + training_type: trainingType, + adapter_rank: adapterRank, + adapter_alpha: adapterAlpha, + adapter_dropout: adapterDropout, + target_modules: targetModules.trim() ? targetModules.split(',').map(s => s.trim()) : [], + dataset_source: datasetSource, + dataset_split: datasetSplit, + num_epochs: numEpochs, + batch_size: batchSize, + learning_rate: learningRate, + gradient_accumulation_steps: gradAccum, + warmup_steps: warmupSteps, + max_steps: maxSteps, + save_steps: saveSteps, + weight_decay: weightDecay, + gradient_checkpointing: gradCheckpointing, + optimizer, seed, + mixed_precision: mixedPrecision, + max_seq_length: maxSeqLength, + eval_strategy: evalEnabled ? (evalStrategy || 'steps') : 'no', + eval_steps: evalSteps, + eval_split: evalSplit, + eval_dataset_source: evalDatasetSource, + eval_split_ratio: evalSplitRatio, + extra_options: Object.keys(extra).length > 0 ? extra : {}, + reward_functions: rewardFunctions.length > 0 ? rewardFunctions : undefined, + } + } + + const applyFormConfig = (config) => { + if (config.model != null) setModel(config.model) + if (config.backend != null) setBackend(config.backend) + if (config.training_method != null) setTrainingMethod(config.training_method) + if (config.training_type != null) setTrainingType(config.training_type) + if (config.adapter_rank != null) setAdapterRank(Number(config.adapter_rank)) + if (config.adapter_alpha != null) setAdapterAlpha(Number(config.adapter_alpha)) + if (config.adapter_dropout != null) setAdapterDropout(Number(config.adapter_dropout)) + if (config.target_modules != null) { + setTargetModules(Array.isArray(config.target_modules) ? config.target_modules.join(', ') : String(config.target_modules)) + } + if (config.dataset_source != null) setDatasetSource(config.dataset_source) + if (config.dataset_split != null) setDatasetSplit(config.dataset_split) + if (config.num_epochs != null) setNumEpochs(Number(config.num_epochs)) + if (config.batch_size != null) setBatchSize(Number(config.batch_size)) + if (config.learning_rate != null) { setLearningRate(Number(config.learning_rate)); setLearningRateText(String(config.learning_rate)) } + if (config.gradient_accumulation_steps != null) setGradAccum(Number(config.gradient_accumulation_steps)) + if (config.warmup_steps != null) setWarmupSteps(Number(config.warmup_steps)) + if (config.max_steps != null) setMaxSteps(Number(config.max_steps)) + if (config.save_steps != null) setSaveSteps(Number(config.save_steps)) + if (config.weight_decay != null) setWeightDecay(Number(config.weight_decay)) + if (config.gradient_checkpointing != null) setGradCheckpointing(Boolean(config.gradient_checkpointing)) + if (config.optimizer != null) setOptimizer(config.optimizer) + if (config.seed != null) setSeed(Number(config.seed)) + if (config.mixed_precision != null) setMixedPrecision(config.mixed_precision) + + // max_seq_length is a top-level field or sits inside extra_options. + if (config.max_seq_length != null) { + setMaxSeqLength(Number(config.max_seq_length)) + } else if (config.extra_options?.max_seq_length != null) { + setMaxSeqLength(Number(config.extra_options.max_seq_length)) + } + + // Eval options: the strategy tells whether evaluation was on. + const restoreEval = (strategy, steps, split, src, ratio) => { + if (strategy != null && strategy !== 'no') { + setEvalEnabled(true) + setEvalStrategy(strategy) + } else if (strategy === 'no') { + setEvalEnabled(false) + } + if (steps != null) setEvalSteps(Number(steps)) + if (split != null) setEvalSplit(split) + if (src != null) setEvalDatasetSource(src) + if (ratio != null) setEvalSplitRatio(Number(ratio)) + } + restoreEval(config.eval_strategy, config.eval_steps, config.eval_split, config.eval_dataset_source, config.eval_split_ratio) + // extra_options overrides the top level when it carries them. + const eo = config.extra_options + if (eo) restoreEval(eo.eval_strategy, eo.eval_steps, eo.eval_split, eo.eval_dataset_source, eo.eval_split_ratio) + if (eo?.save_total_limit != null) setSaveTotalLimit(Number(eo.save_total_limit)) + + // liquid-audio extras; they are also kept out of the free-form list below. + if (eo?.voice != null) setLiquidAudioVoice(String(eo.voice)) + if (eo?.val_dataset != null) setLiquidAudioValDataset(String(eo.val_dataset)) + + if (config.extra_options && typeof config.extra_options === 'object') { + setExtraOptions(Object.entries(config.extra_options) + .filter(([k]) => !EXTRA_HANDLED.includes(k)) + .map(([key, value]) => ({ key, value: String(value) }))) + } + setRewardFunctions(Array.isArray(config.reward_functions) ? config.reward_functions : []) + } + + const handleExportConfig = () => { + const url = URL.createObjectURL(new Blob([JSON.stringify(getFormConfig(), null, 2)], { type: 'application/json' })) + const a = document.createElement('a') + a.href = url + a.download = 'finetune-config.json' + document.body.appendChild(a) + a.click() + document.body.removeChild(a) + URL.revokeObjectURL(url) + } + + const handleImportConfig = () => { + const input = document.createElement('input') + input.type = 'file' + input.accept = '.json' + input.onchange = (e) => { + const file = e.target.files[0] + if (!file) return + const reader = new FileReader() + reader.onload = (ev) => { + try { + applyFormConfig(JSON.parse(ev.target.result)) + setSelectedJob(null) + setError('') + } catch { + setError(t('fineTune.badConfig')) + } + } + reader.readAsText(file) + } + input.click() + } + + // Put a job's setup back in the form. `fixes` are the changes ticked on a + // memory failure: they are applied to this copy and nothing starts until the + // person presses Start. + const handleUseConfig = (job, fixes) => { + applyFormConfig(job.config || job) + setResumeFromCheckpoint('') + if (fixes?.batch) setBatchSize(1) + if (fixes?.checkpointing) setGradCheckpointing(true) + setSelectedJob(null) + } + + const handleResumeFromCheckpoint = (checkpoint) => { + if (!selectedJob) return + applyFormConfig(selectedJob.config || selectedJob) + setResumeFromCheckpoint(checkpoint.path) + setShowAdvanced(true) + setSelectedJob(null) + } + + // Baseline for the unsaved-changes guard: lazily taken on first render. + if (initialConfigRef.current === null) initialConfigRef.current = JSON.stringify(getFormConfig()) + const dirty = JSON.stringify(getFormConfig()) !== initialConfigRef.current + // Unsaved edits: the shell asks before it replaces or leaves the page. + const unsaved = dirty && !selectedJob && !loading + useEffect(() => { ctx.markDirty(unsaved) }, [unsaved]) + useEffect(() => () => ctx.markDirty(false), []) + + const checks = useMemo(() => fineTuneChecks({ + form: { + model, datasetSource, datasetFileName: datasetFile?.name || '', backend, batchSize, maxSeqLength, gradAccum, + gradCheckpointing, trainingType, trainingMethod, rewardCount: rewardFunctions.length, + resumeFrom: resumeFromCheckpoint, hfToken: hfToken.trim(), + }, + facts, + backends: { fineTune: backends }, + }), [model, datasetSource, datasetFile, backend, batchSize, maxSeqLength, gradAccum, gradCheckpointing, trainingType, trainingMethod, rewardFunctions.length, resumeFromCheckpoint, hfToken, facts, backends]) + const stop = blocked(checks) + const warnings = warned(checks) + const firstMissing = checks.find(c => c.tone === 'fail') + + const steps = [ + { key: 'setup', label: t('steps.setup') }, + { key: 'check', label: t('steps.check') }, + { key: 'run', label: t('steps.run') }, + { key: 'result', label: t('steps.result') }, + ] + let current = stop ? 0 : 1 + let failed = false + if (selectedJob) { + if (selectedJob.status === 'failed') { current = 2; failed = true } else if (TERMINAL.includes(selectedJob.status)) current = 3 + else current = 2 + } + + const kindWords = trainingType === 'full' ? t('form.summaryFull') : t(`form.summary_${trainingType}`) + const modelShort = model.split('/').pop() || model + + return ( +
+ {t('fineTune.title')} {t('landing.experimental')}} + supporting={t('fineTune.lede')} + actions={( +
+ {selectedJob && ( + + )} + +
+ )} + /> + + + {error && ( +
+ +

{error}

+
+ )} + + {selectedJob ? ( + <> + + + + ) : ( +
+ {resumeFromCheckpoint && ( +
+ + {t('fineTune.resuming')} {resumeFromCheckpoint} + +
+ )} + +
+ + setModel(e.target.value)} placeholder="TinyLlama/TinyLlama-1.1B-Chat-v1.0" required /> + +
+ +
+
+ + setDatasetSource(e.target.value)} placeholder="tatsu-lab/alpaca" /> + + + setDatasetSplit(e.target.value)} placeholder="train" /> + +
+
+ setDatasetFile(e.target.files[0] || null)} accept=".json,.jsonl,.csv" /> + + {datasetFile ? ( + + {datasetFile.name} + + + ) : ( + {t('form.uploadHint')} + )} +
+
+ +
+
+ {t('form.kind')} + {[['adapter', isAdapter], ['full', trainingType === 'full']].map(([id, on]) => ( + + ))} +
+
+ + setNumEpochs(Number(e.target.value))} min={1} /> + + + setBatchSize(Number(e.target.value))} min={1} /> + + + { + setLearningRateText(e.target.value) + const parsed = Number(e.target.value) + if (!isNaN(parsed) && parsed > 0) setLearningRate(parsed) + }} + placeholder="5e-5" + /> + +
+ +
+ + {showAdvanced && ( +
+
+ + + + + + + {isAdapter && ( + + + + )} + {isAdapter && ( + <> + + setAdapterRank(Number(e.target.value))} min={1} /> + + + setAdapterAlpha(Number(e.target.value))} min={1} /> + + + setAdapterDropout(Number(e.target.value))} min={0} max={1} step={0.05} /> + + + )} + + setGradAccum(Number(e.target.value))} min={1} /> + + + setWarmupSteps(Number(e.target.value))} min={0} /> + + + setMaxSeqLength(Number(e.target.value))} min={64} /> + + + + + + setMaxSteps(Number(e.target.value))} min={0} /> + + + setSaveSteps(Number(e.target.value))} min={0} /> + + + setSaveTotalLimit(Number(e.target.value))} min={0} /> + + + setWeightDecay(Number(e.target.value))} min={0} step={0.01} /> + + + setSeed(Number(e.target.value))} min={0} /> + + + + +
+ {isAdapter && ( + + setTargetModules(e.target.value)} placeholder="q_proj, v_proj, k_proj, o_proj" /> + + )} + + + setHfToken(e.target.value)} placeholder="hf_..." autoComplete="off" /> + + +
+
+
+ {evalEnabled && ( +
+ + + + + setEvalSteps(Number(e.target.value))} min={0} /> + + + setEvalSplit(e.target.value)} placeholder="validation" /> + + + setEvalDatasetSource(e.target.value)} placeholder={t('form.evalDatasetPlaceholder')} /> + + + setEvalSplitRatio(Number(e.target.value))} min={0.01} max={0.5} step={0.01} /> + +
+ )} +
+ + {trainingMethod === 'grpo' && ( +
+

{t('form.rewards')}

+

{t('form.rewardsHint')}

+ {BUILTIN_REWARDS.map(builtin => { + const selected = rewardFunctions.find(rf => rf.type === 'builtin' && rf.name === builtin.name) + return ( +
+ + {selected && builtin.params.map(param => ( + + setRewardFunctions(prev => prev.map(rf => (rf.type === 'builtin' && rf.name === builtin.name ? { ...rf, params: { ...(rf.params || {}), [param.key]: e.target.value } } : rf)))} + /> + + ))} +
+ ) + })} + {rewardFunctions.filter(rf => rf.type === 'inline').map((rf, idx) => ( +
+
+ {rf.name} + +
+ 
{rf.code}
+
+ ))} + {showAddCustomReward ? ( +
+ + setCustomRewardName(e.target.value)} placeholder="my_custom_reward" /> + + +