mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 00:59:28 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(addons): installable add-ons foundation and the Tables add-on (#12702)

Browse Source 
* feat(addons): add the add-on runtime, navigation and first-party add-ons

Add the add-on runtime to LocalAI: manifests, the host operation broker,
admin gating by account role, the add-on UI bundle builder and the SDK
for add-on pages. Move the navigation to Chat, Models, Add-ons and
Settings, with pinned add-ons in the sidebar and an Add-ons drawer.

Ship the first-party add-ons (voice, face, fine-tuning, quantization) as
packages for linux/amd64, linux/arm64 and darwin/arm64. Intel macOS
(darwin/amd64) is enabled in the platform files.

Squashed from the review branch. The tree is unchanged; the pre-squash
history is kept on a local backup branch.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gosec): tighten file modes and annotate false positives

Write job state, configs and imported models with mode 0600, clean
the paths that are read from disk, and mark the route tables, the
header name and the retry jitter as reviewed false positives.

Assisted-by: Claude Code:claude-sonnet-5-5 gosec
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): keep API keys off recovery mutations and same-site cookies

POST /api/addons/dispatch forwarded the recovery and update
mutations to the owner for any administrator key, which skipped the
browser-session rule of their own routes. Refuse those actions
there. Also apply the same-origin rule to the legacy token cookie,
not only the session cookie, because the global CSRF check lets
same-site requests through.

Do not fail the start of LocalAI when an add-on library cannot be
refreshed; log it and keep the installed add-ons up.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): refuse a reinstall that would migrate retained data unbacked

An upgrade takes a verified backup before the new package migrates
the data. A reinstall over retained data has no running generation
to stop, so the new package would migrate it without one. Refuse a
reinstall whose data format differs from the retained one.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): release the manager lock while a release is fetched

Submit held the state lock across the catalog lookup and the package
download, so the apps list and status reads stalled for the whole
install. Serialize Submit and Reconcile with their own lock and drop
the state lock around the fetch only.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): kill an add-on only after three missed health probes

One slow 100 ms probe killed a busy add-on and dropped the work it
was serving. Require three misses in a row.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): move settings under the operate hub

The Operate hub already has Settings as a tab, so the sidebar no longer shows a separate Settings entry. The /app/settings route stays, and the hub tab still reaches it. The navigation test now expects Chat, Models and Add-ons as the primary items.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): accept same-host browser requests that carry no fetch metadata

Browsers do not send Sec-Fetch-* headers over plain HTTP on an address
that is not localhost, so every add-on request from a LAN address was
refused as cross-site. Keep refusing a Sec-Fetch-Site value other than
same-origin. When the header is absent, accept the request only if its
Origin, or failing that its Referer, names the same host and port as
the request.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): show the server's reason when an add-on request is refused

A 403 or 400 answer showed a generic message and hid the reason in the
response body, so a refusal by the browser guard could not be told from
a missing permission. Show error.message from the body as the server
sent it, and keep the generic text when the body has none.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): run the add-on owner in the normal binary

The owner was only enabled when an embedding host set the add-on config,
so a normal run answered every add-on request as unavailable.

Add --addons-enabled (default true) and --addons-ownership-mode
(default named-user, or local-owner; anything else stops startup).
The node is its own owner: state lives under the data path, and the
carrier uses loopback keys made at each start and kept in memory.
--addons-enabled=false turns the owner off completely.

Named-user mode needs the account database. With authentication off
the owner stays off and a warning says how to use local-owner.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): keep the loopback owner address stable across restarts

The single-node loopback transport took a new ephemeral port at each
start. The owner stores its address in its identity file and refuses to
start when the address differs, so every start after the first failed
with "invalid addon owner configuration".

Choose the port once, keep it in the add-on state root (loopback-port,
mode 0600) and reuse it. If the port is taken at a later start, stop
with an error that names the port and --addons-enabled=false, because
another port would break the identity check. The keys and certificate
stay in memory.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): let add-on pages keep the host modules they were written against

A page written for the web UI imports React, the router, i18next and a
few shared components. The add-on builder refuses a bundle that holds a
host file, so moving such a page into an add-on meant rewriting it.

Add a compat mode to the shared builder. It answers those imports from a
directory of stand-ins that read the SDK, bundles the dynamic import of
the page code into the one output file, and collects the style sheets the
page imports. A relative import that is not in the add-on's own source
resolves to the file at the same path under the compat directory.

Widen the SDK for these pages, without a version change: more host
components (spinner, modal, lightbox, model picker) and hooks (models,
resources, voice profiles, operations, MCP client), a style on icons,
the icon names the pages draw, sdk.navigation.openApp to open a page of
the app (only addresses under /app, and the way back as the only state)
and sdk.portal. The shell passes the query string to the page.

The stand-ins cover the router (the tab in the path, the model in the
query), the translator over the host locale files, the core API calls
through sdk.core.fetch, and byte copies of the pure helpers, with tests
that keep the copies equal to the originals.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): ship Studio and Talk as add-ons instead of built-in pages

Studio and Talk were compiled into the web bundle and always listed in
the Add-ons drawer. They are add-ons now, like Voice, Face, fine-tuning
and quantization: a person installs and removes them, the drawer lists
them only while they are installed, and they are packaged the same way.

Move the pages, their components, hooks and helpers into addons/studio
and addons/talk. Each has a manifest, a UI bundle, a release platform
matrix and a catalog template, and is built by the compat mode of the
shared builder. The core API stays where it is: the pages call it with
the signed-in person's session. The history keeps its browser storage
keys. The unit tests of the moved helpers move with them.

In the core, drop the two pages, the Create group of the sidebar (Chat
joins Home and Models in the old layout), the built-in catalog entries
(the catalog is empty) and the talk locale namespace. The old addresses
(/app/studio, /app/talk and the generator links) redirect to the add-on
pages, with the model as ?model=. The Home /studio command is offered
only while the Studio add-on is installed and opens it.

The release workflow and its tests now list seven add-ons.

Assisted-by: Claude Code:claude-sonnet-5-5 go node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): run the Studio and Talk specs against the add-on bundles

The pages are add-ons now, so a spec that draws one installs the real
built bundle first: the apps list names it and its module and style
sheet are served from the build output, the way the add-on owner serves
them. studio-addon.js holds that helper, and the Studio and Talk
fixtures call it.

Update the specs for the new addresses (/app/addons/studio/studio, the
model as ?model=), the session permission the add-on route needs, the
legacy redirects, the Home /studio command that appears only while the
add-on is installed, and the sidebar without a Create group.

Add specs for the add-ons themselves: every tab draws with no script
error, the shell titles the page once, the old addresses say so when the
add-on is not installed, a button of the page reaches a core page with
the way back, and the drawer lists both only after they are installed.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): centre the icon of a built-in tile in its icon box

The drawer drew the icon of a built-in tile (Studio's palette was one)
in a bare 40px box. Only the placeholder box was laid out as a centred
grid with the 20px size, so the real icon sat in the top-left corner at
the size of the page text.

Give the built-in tile the same classes as the placeholder box, and
check in a browser that the icon is centred.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(addons): describe Studio and Talk as add-ons

Replace the built-in list (now empty) with the Studio and Talk add-on
sections, the new targets of the old links, the Home command, and the
seven packaged add-ons. Describe the SDK additions the pages use:
components, hooks, icons, openApp and portal. Say on the Studio page
that it is installed from the drawer.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): click the shell title in the Studio slash-key spec

The add-on shell titles the page, and the Studio overview no longer draws
a second title, but the generator tabs keep theirs, so a bare h1 locator
matched two elements. Click the first, which is the shell's.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): read the loopback port file through os.Root

The persisted port file was read with a path built from the add-on
root, which gosec reports as a file inclusion through a variable. Open
it through os.Root so the name cannot leave the root, even by a
symbolic link, and require a regular file with no group or other
access. A looser file now fails closed instead of being trusted.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): expect Settings under the Operate hub, not the sidebar

Settings moved out of the sidebar into the Operate hub. The navigation
spec still expected a sidebar link. Assert that the sidebar has no
Settings entry, and that the Operate hub opens Settings as a tab.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): drop Settings from the 404 places

Settings is a tab of the Operate hub and not a destination of its own,
so the 404 page does not list it. Assert that it is absent.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): drive the media history and TTS specs through the Studio add-on

The image, video, sound and speech generators are pages of the Studio
add-on now, and the old addresses redirect to it. Install the add-on
with the Studio fixtures, open its pages and use its compose card, so
the specs run against the page that exists.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): count the first-run steps, not every list item

The recommended models are a list inside the second step, so counting
all list items read 3 only while the models were still loading. Count
the step elements, which does not depend on the load order.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): fail the node refresh by state, not by call count

The spec served the list to the first request and failed the rest. The
first request is not always the one from the page, so the page could
get the failure first and show no list. Serve the list until the page
shows it, then fail the refreshes.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): default to local-owner when auth is off

A server without authentication has no accounts, so named-user left
the add-on owner off. Resolve the unset mode in one place: local-owner
when authentication is off, named-user when it is on. A flag or
environment value still wins, and an unknown value is still refused.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): register an official add-on library by default

Add gallery/addons/index.yaml, a YAML index of the four first-party
add-ons (fine-tuning, quantization, voice, face) in the library format.
The entries come from addonpkg catalog over locally built linux/amd64
packages, so the digests pin those images.

On the first start, when the owner has no sources.json, register the
raw GitHub URL of that index on master as the "official" library and
write the file. The file is then the record: a library the admin
removes stays removed, and an existing file is never changed.
--addons-official-library=false skips the default.

Assisted-by: Claude Code:claude-sonnet-5-5 go node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(addons): document the ownership default and the official library

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): list Studio and Talk in the first-party index

Add the Studio and Talk entries to gallery/addons/index.yaml next to
the four existing ones. They come from addonpkg catalog over locally
built linux/amd64 packages, so the digests pin those images.

Assisted-by: Claude Code:claude-sonnet-5-5 go node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): expect Studio and Talk in the official index

The official index now lists six add-ons. Update the test that
decodes it so it accepts Studio and Talk.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): add a real icon to each first-party add-on

Each icon is a 256 by 256 PNG: a rounded square in one colour of the
UI kit palette with a white outline glyph from the kit sprite. Fine-tuning
uses sliders, quantization layers, voice waveform, face user, studio
palette, talk phone and tables database. They were rendered with headless
Chrome from the sprite paths.

Assisted-by: Claude Code:claude-sonnet-5-5 chrome
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): ship a committed icon.png instead of the generated square

The shared UI builder and the Tables builder copy ui/icon.png from the
add-on source tree to the package unchanged when the file exists. They
write the generated placeholder only when it does not. Add a fixture with
an icon and a test for both cases.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): list the tables add-on in the first-party index

Add the Tables entry to gallery/addons/index.yaml next to the six
existing ones. It comes from addonpkg catalog over a locally built
linux/amd64 package, so the digest pins that image. The official index
test now expects seven add-ons.

Assisted-by: Claude Code:claude-sonnet-5-5 go node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): make the Voice add-on one page with tabs

The Voice add-on had four pages: recognition, library, create and
speech, and the drawer listed two of them. It now has one page, "voice",
and one drawer entry. Recognition, Library, Text to speech and From a
recording are tabs of that page, drawn with the same tab strip the
recognition page already used. The tab is the first path segment
(voice/library), as in the Studio add-on, and the form that adds a voice
is a view of the Library tab (voice/library/new).

The old page names stay declared in the manifest. Each one redirects to
its tab and keeps the query string, so saved links still open.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): send the old voice routes to the Voice tabs

/app/voice, /app/voice-library and /app/voice-library/new now open the
matching tab of the Voice add-on page while the add-on is installed and
running, and the core pages otherwise. The moved-page helper compares
the first segment of the target with the declared pages, so a target
can name a tab. The docs and the navigation spec list the new addresses.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): cover the Voice tabs and the old page redirects

The add-on unit tests render the tabs instead of the four pages. The
Playwright spec checks the tab strip, the address of each tab, the
missing Library tab for a member, and that the old page addresses of the
add-on redirect to their tab with the query string kept. The migration
table test follows the new targets.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(studio): keep the model menu inside a narrow screen

The Home model picker sizes its menu from the viewport, and assumes the
picker sits at the page gutter. In the Studio composer card the field is
further in, so on a 390px screen the menu of the Sound expansion picker
ended 7px past the right edge. Size the menu from the field it opens
under instead.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): build the add-on bundles once for the whole run

Each worker built the Studio and Talk bundles into the same dist
directory, and the build empties that directory first. A worker that
started, or restarted after a failure, removed the files another worker
was serving, so a Studio spec saw "the style sheet of the add-on could
not be loaded". Build once in a global setup before the workers start.
A run without the setup still builds per worker.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): register the worker when the add-node spec says so

The mocked roster added the worker on the third read of /api/nodes. The
shell and the page both read it, and which one asks first depends on
load, so the page could take the new worker as part of the roster it
opened with and never show it as found. The worker now joins after the
page shows it is listening.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): hold the gallery answer until the skeleton is seen

The spec delayed the answer by a fixed 800 ms and then looked for the
skeleton. The assertion polls at growing intervals, so on a slow machine
it could step over that window. The answer now waits until the skeleton
has been seen, and the spec then checks that the skeleton goes away.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): give the PDF tag the time the other PDF specs have

pdf.js and its worker load on first use. The other specs in this file
wait up to ten seconds for the result of that load. The first one waited
five, and missed the tag on a busy machine.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): build the Face and Fine-tuning bundles in the global setup

The navigation spec serves the built Fine-tuning bundle, but nothing
built it, so two specs failed on a fresh checkout when the dist folder
was missing. The spec also built the Face bundle in a hook that every
worker runs, and a build rewrites the folder the other workers serve.

The global setup now builds Studio, Talk, Face and Fine-tuning once,
before the workers start. The spec builds the two bundles itself only
when it runs without the global setup.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): fix the Home answers in the stale pin spec

The spec asserts that the page shows no alert. The Home page raises its
own alert when the model list fails or a staging operation has failed,
and both answers come from the shared test server. A slow reply or a
failed install left there counted as an alert about the pin.

The spec now fixes both answers, so the alert count tells only about the
pin. The assertions are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): read the server address in the add-node spec from the page

The spec expected the register address to carry port 8089, so it failed
on any other port. It now expects the origin the page was opened from.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): give the navigation routes spec its own catalog

No page is built in today, so the catalog is empty and the three specs
that expect the studio and talk entries failed on the list and on the
pin seed. They passed only while Studio and Talk were built in.

The spec now sets a catalog of two ungated entries for each example and
restores the old one afterwards. The assertions are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(cli): report a failed write of the MLX hostfile

The tunnel callback ignored the error of os.WriteFile and set the
hostfile variable even when the file was not written. It now logs the
error and leaves the variable alone.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): discard the close error of the held listener

errcheck flags the deferred Close of the listener that keeps the port
busy. The test has no use for that error, so the defer drops it
explicitly.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): move the add-on tool tests to Ginkgo

The fine-tuning, quantization and add-on host library tests used
t.Errorf, t.Fatal and t.Run, which the forbidigo rule rejects. They now
use Describe, It and Gomega, with a suite bootstrap in each package. The
cases and the assertions are the same.

The tables in the host client and the tool schema tests keep one entry
per case.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): move the host operation tests to Ginkgo

The hostops tests used the testing package calls that forbidigo rejects.
They now run as Ginkgo specs under one suite bootstrap. Every check and
every per-operation case is kept.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): move the pin store tests to Ginkgo

The pin service, file store, sweep and shared store contract tests used
the testing package calls that forbidigo rejects. The store contract is
now a function that registers its specs for each store, so the file
store, the memory-only store and the database store run the same cases.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): move the remaining add-on tests to Ginkgo

The built-in catalog, tool arguments, loopback carrier, host operation
contract, packaging, application, config, CLI and endpoint tests used
the testing package calls that forbidigo rejects. They now run as Ginkgo
specs. The endpoint tests also drop the unchecked Close errors that
errcheck reports when the auth tag is on.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): wait for the targeted Reinstall button before the install click

The "generic deletion blocks targeted install at confirmation" spec
dispatches a popstate to set the target node, then clicks Reinstall with
a script click. The click can run before the targeted page has rendered
or while the button is still disabled, so no install request is sent and
the poll on the install count times out. It failed 1 run in 10 with two
workers and on every retry in CI.

The spec now waits for the button to be enabled first. The assertions
are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(addons): refresh the voice package digest in the first-party index

The Voice add-on became one page with tabs and got a new icon, so the
digest in gallery/addons/index.yaml no longer matched the build. Render
the entry again with addonpkg catalog over a locally built linux/amd64
package; the main page is now voice. The other entries are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 go node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): wait for the target node's Reinstall button after the popstate

The "generic deletion blocks targeted install at confirmation" spec
dispatches a popstate to set the target node, then waits for a button
named Reinstall and clicks it. Until React renders the targeted page,
the page still draws the generic Reinstall button. It is enabled and has
the same name, so the wait passed on it, the click started a global
install, and no node install request was sent. The poll on the install
count timed out with the confirm dialog still open.

With the CPU throttled to 8x, the spec failed on 20 of 20 runs, and the
button it clicked had the title "Reinstall". The spec now waits for the
button titled "Reinstall on <node>". With the throttle it passes 30 of
30 runs. The assertions are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): give the Studio hand-off spec a budget for the cold bundle load

The "a result made through a hand-off records which result it came from"
spec opens the Studio add-on straight from an address, twice. Each is a
cold load: the page shows "Loading" until the add-on module has run, and
the next assertion had the default five seconds. The load takes about
two seconds on an idle run and seven to ten under CPU load, so the spec
failed on a loaded machine with the hand-off note not yet drawn.

With the CPU throttled to 6x and eight busy loops running, it failed on
21 of 30 runs. The first assertion after each load now waits up to 20
seconds, and the spec then passed 30 of 30 under the same load. The
assertions are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): carry the identity page styles in the Face add-on

The Face page is drawn with the shared identity views, but its style sheet
held only the clip and error rules. The page, card, model list, known-faces
table, enrol sheet and distance scale classes had no rules. They looked
right only when the host had already loaded the style sheet of its own
Faces page, which a visit through the add-on drawer does not do. Opened
that way, the cards had no gap, the model list showed bullets, and the
install buttons ran into the names.

The Face sheet now holds the same identity rules as the core sheet, so the
page looks the same whichever way it is opened.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): scope the narrow-screen identity rules of the Voice add-on

The rules for narrow screens (single-column pairs, smaller tool padding,
compact rank rows) were copied into the Voice style sheet without their
media query, so they applied at every width. They now sit in a
max-width 720px query, and the reduced-motion rule in its own query, as in
the core sheet.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): drop the repeated title from the Voice and Face pages

The add-on shell already titles the page, so the recognition pages showed the name twice: "Voice" above "Voices", "Face" above "Faces". Keep the supporting line and the model picker, as the Studio page does, and remove the unused title strings.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): replace the Pinned sidebar section with an Apps entry

The sidebar no longer lists pinned add-ons. It shows one Apps entry that
opens the add-ons drawer, with a Show all link below it that opens the
same drawer. Pins are used from the drawer, where a pinned tile keeps
its pin and move buttons. The server pins API is unchanged.

Remove the PinnedNav component and its locale title, rename the
items.addons label to Apps, and update the e2e specs and the add-ons
documentation to match.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): assert the single shell title on the Voice and Face pages

The add-on shell now shows the one title, "Voice" or "Face", and the page no longer repeats it. The tests still looked for the h1 "Voices" and "Faces". Assert that the add-on page has exactly one h1 and that it is the shell title. The core pages keep their own "Voices" and "Faces" titles.

Assisted-by: Claude Code:claude-sonnet-5-5 playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): drop the repeated title from the Voice Library and Text to speech pages

The add-on shell already titles the page "Voice", so these two tabs showed a second heading below it ("Speech voices", "Speak text"). Keep the supporting line, as the Studio and recognition pages do, and remove the unused title strings. The e2e spec now checks that each tab has exactly one h1, the shell title.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): list pinned apps under Apps in the sidebar

A pinned app now shows as an indented child of the Apps entry, with its icon and title, in pin order. Show all stays the last child. The pins come from GET /api/addons/pins through the navigation context, and the apps from the add-on registry, which the drawer feeds on every list read, so the links follow the drawer. A pin without an installed app is not drawn, nothing extra shows when no app is pinned, and the children are hidden when the sidebar is collapsed to the rail.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(addons): say that pinned apps are listed under Apps in the sidebar

The sidebar lists the pinned apps between Apps and Show all, in pin order. The pins section and the navigation section said that pins do not appear in the sidebar.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): remove the Show all link under Apps in the sidebar

The link repeated the Apps entry, which opens the same drawer, and looked redundant when no app was pinned. Apps is now the only add-on entry; pinned apps still list under it in pin order. The link, its locale key and the style comment are gone, and the e2e spec and the add-ons docs now say that Apps is the only entry.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): show a first-run card on Apps when nothing is installed

An administrator who opens Apps with no add-on installed now sees a
short card with three steps: choose a library, browse and install,
start the add-on. Its button opens the Browse tab. The card is hidden
once an add-on is installed and for anyone who cannot install.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): hide the pinned apps in the icon rail

When the window is narrower than 1024 px the sidebar shrinks to the icon rail by CSS alone, without the collapsed class. The pinned app links were still drawn there, with the indent meant for the full sidebar, so their icons sat to the right of the rail and were cut off. Hide the pinned app links in both rail states. The Apps entry keeps its centred icon. The e2e spec checks that the rail shows no pinned app and that every icon is centred at 1280 px collapsed and at 1000 px.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): stop the backend redirect test racing the installed list

The test reads the redirected URL right after the path changes. The
Backends page then loads the installed list and drops a selected backend
that is not installed, so the backend parameter was sometimes already gone
when the test read it. The mock server has no llama-cpp, and about 15 of
40 repeated runs failed on the backend assertion. The redirect itself was
correct, including show_all.

Serve the selected backend in the installed list, so the page keeps the
selection and the assertions stay as strict as before.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): accept a same-host Referer or Origin for human confirmation

Browsers send no Sec-Fetch-* headers over plain HTTP on an address that
is not localhost, so creating a resource from a LAN address failed with
"Creating a resource requires your own browser session".

When Sec-Fetch-Site is present it must still be same-origin with an
empty destination, as before. When it is absent, Origin or, failing
that, Referer must name the request host and port. A destination or
mode that is present must still be the expected one. This matches the
rule in BrowserProtection. The credential, token cookie, API key and
tool-context checks are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): keep hub entries off the Apps route in the sidebar

The Build hub still owned /app/addons through its legacy Add-ons tab,
so the sidebar lit both Apps and Build on the Apps page. A tab whose
gate is closed now owns no route when the sidebar asks, so only the
Apps entry is active there. The hubs keep their own routes, and the
legacy navigation still gives /app/addons to the Build hub.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): label the pin control on app tiles and in the detail view

The pin was a small icon-only button and hard to find. Each app tile now
shows a button reading Pin, or Pinned with a filled pin, and the detail
view has the same control in its header. Both toggle the same stored pin.

The Apps grid lists pinned apps first, in pin order, then the others.
The sidebar list and the server pins API are unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): check the Voice From a recording tab has one title

The Library and Text to speech tabs already assert that the add-on shell
title is the only h1. Do the same for the From a recording tab, so a
heading added there later fails the navigation spec.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): stop the add-on shell drawing a title over add-on pages

The app decides what its page shows. The shell drew the add-on name as a
header above every add-on page, so an add-on could not choose its own
title. The shell now draws no header on a live, loading or frozen
add-on page; each add-on titles its own pages. The unavailable and
missing-page states keep the name, because no add-on page is drawn there.

The runtime and shell specs now check that the shell adds no page title.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): let each add-on draw its own page title

The shell no longer titles add-on pages, so each add-on draws one h1 per
page. Voice titles its tabs "Voices" (Recognition and From a recording),
"Speech voices" (Library) and "Speak text" (Text to speech); "Add a speech
voice" already had its own. Face draws "Faces", Studio "Studio" and Talk
"Talk". Tables promotes the database list and database name from h2 to h1.
Fine-tuning and Quantization already drew a title.

The add-on unit tests and the navigation and Studio specs now assert
exactly one h1 per page, with the page's own title.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): add a kind to library entries and read unknown kinds tolerantly

A library entry now has an optional kind: addon (the default when it is
absent), backend or model. Every index written before kinds existed stays
valid and its entries are add-ons.

A reader keeps the entries of the kinds it knows and skips the others
without reading their fields, so a newer kind never makes an older host
refuse the whole index. The refresh result and the library state report
how many entries were skipped and of which kinds, and the refresh logs a
warning. A kind that is not a plain kind name still makes the index
malformed. Search lists add-ons unless the caller asks for a kind, and
Resolve returns add-ons only, so an add-on install cannot be pointed at
a backend image.

A library policy may name the signer with an anchored identityRegex
instead of an exact identity, so builds from master and from tags both
verify. An add-on install refuses a policy that carries one.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(backends): install a backend from a library entry

A library can now list backends. An entry of kind backend names the
image of the backend gallery by digest, and POST /backends/library/install
installs it on the local node. The gallery stays the source of truth and
its install path is the one used: the library resolves the entry, takes
the signature policy from the library, pins the image by digest and calls
the same install as the gallery. Installing a name that came from a
gallery, another library or by hand is refused and changes nothing.

The signature policy of a backend library names the signer with an
anchored identityRegex, so images signed by the merge workflow on master
and on tags both verify. A pattern that is not anchored is refused.

An install is bounded by a size limit per kind: add-ons keep 512 MiB and
a backend image defaults to 8 GiB, set with --addons-backend-max-bytes.
The limit reads the manifest after the signature check and before any
layer is fetched. The metadata of an installed backend records the
library it came from.

scripts/backends/cmd/backendcatalog renders entries from
backend/index.yaml and pins them against the registry. The pilot is the
CPU build of local-store, in gallery/backends/index.yaml. The library is
registered on the first start only with --addons-backend-library, which
is off by default.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): show the kind of library entries and install backends from them

The Browse tab lists the entries of every kind this host reads and marks
each one as an add-on or a backend. A backend installs through the
backend route and not through the add-on lifecycle: the confirmation
says that the backend gallery is not changed and that a backend runs
native code, and a refusal of the host, such as a name installed from
another origin, is shown with its own reason. The details give the kind,
the alias and the signer pattern.

The library editor can name the signer with a pattern. Without this a
save would drop the pattern of a backend library. The Libraries tab
reports the entries a refresh skipped because their kind is not read.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(addons): describe backends in the library

Document the entry kinds, how a reader skips a kind it does not know,
the backend entry format, the signer identity pattern, the install
route, the size limit of each kind, the pilot library and the command
that renders it. The backend page links to it.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(mcp): install a backend from a library entry

The backend library route installs a backend from an add-on library
entry, and the admin REST surface and the MCP catalog must stay in step.
Add the install_library_backend tool with an in-process and an HTTP
client, the route map entry, the safety-rule listing and a skill recipe.

Listing library entries stays on the add-on routes, which the assistant
does not expose, so no list tool is added.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): carry the add-on library keys in every locale

The add-on library and backend library strings existed only in English,
so the other locales had no value for them. Add the missing keys of the
addons namespace to de, es, id, it, ko, pt-BR and zh-CN. The values are
the English text until a translation is contributed.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "feat(mcp): install a backend from a library entry"

This reverts commit 7493cff727.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "docs(addons): describe backends in the library"

This reverts commit 57816ed7e6.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "feat(ui): show the kind of library entries and install backends from them"

This reverts commit 49b2b151b7.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "feat(backends): install a backend from a library entry"

This reverts commit 1d8d3e3421.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* Revert "feat(addons): add a kind to library entries and read unknown kinds tolerantly"

This reverts commit 484813f856.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): drop the backend library keys from the locales

Revert the part of 39f77ae29 that added strings for backend libraries:
the kind labels, the backend install texts, the signer identity pattern
and the skipped entries notice. The add-on library keys stay.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add an add-ons control to the chat header

Add-on tools were on or off only through the /addons command, with a toast
and no state a person could see. The chat header now has an Add-ons button
in the style of the MCP dropdown. It lists each running add-on that ships
tools with a switch, and a badge shows how many are on.

The chat keeps the ids that are on (addonToolIds) next to the older
boolean, and both are saved with the chat. A chat that has only the boolean
means every add-on. The request sends addon_tools alone when every add-on is
on, and adds addon_tool_ids when only some are. /addons turns all on or all
off and uses the same state.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): let a chat request name the add-ons whose tools it uses

addon_tools=true still offers the tools of every add-on the caller may use.
A new addon_tool_ids metadata key, a comma-separated list of add-on ids,
narrows that to some add-ons. The list counts only with the opt-in, and a
list that names nothing offers no tool rather than all of them.

The executor drops the tools of add-ons that are not listed and refuses a
call to one as an unknown tool, so a model cannot reach a tool the person
turned off. Both keys are consumed so they do not reach the backend.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): cover the per-add-on tool list and the chat header control

Go tests cover the list form (the list filters the tools offered and the
calls allowed) and the unchanged boolean form. Unit tests cover the chat
add-on rules. Playwright specs cover the header control: listing only
add-ons that ship tools, toggling each one, the count, /addons in step with
the switches, and the request metadata.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): check each add-on page has one title when not enabled

The shell names an add-on only where the add-on page cannot mount (not
enabled, failed, no UI, bundle not loaded, no such page). Once the page
mounts it draws its own title, including when it says it is not enabled
on this server, and the shell draws none. The shell code already works
this way; this change states the rule beside the unavailable state and
pins it in the specs.

The shell spec now expects exactly one title in the unavailable states.
The navigation spec checks the Face page without a face model (the
add-on title only) and the Face page of a stopped add-on (the shell
title only).

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(addons): upgrade add-ons on their own when the setting is on

A new auto_upgrade_addons setting (flag LOCALAI_AUTO_UPGRADE_ADDONS,
default off) lets the owner upgrade an installed add-on when its library
lists a newer version. It follows the backend setting: a startup flag, a
runtime setting, and the same schedule as the backend update check (a
first pass shortly after start, then every six hours).

Each pass fetches the library of every installed add-on, and for each
available update builds the same pinned upgrade command an administrator
sends and runs it through the same lifecycle path. The channel rules, the
signer identity, the digest checks, the SDK compatibility check and the
backup before the upgrade all apply unchanged. The only difference is the
gate: the owner acts on its own setting, so no administrator is asked.
A refused or failed upgrade changes nothing and is logged; a candidate
that failed is not retried until the library lists another digest.

With the setting off, nothing runs and upgrades stay manual.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add the Upgrade add-ons automatically toggle to Settings

The toggle sits beside Upgrade backends automatically and is off by
default. It saves the auto_upgrade_addons runtime setting. The English
locale gets the label key, and the Settings test fixture lists the new
field.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): cover the automatic upgrade setting and document it

The setting defaults to off in the config, the runtime settings and the
flag, and is read back from the saved settings file. With it on, a real
owner upgrades an installed add-on through the manager; with it off,
nothing changes, and a change of the setting at run time takes effect on
the next pass. The manager specs show the upgrade goes through the same
pinned path as the manual one, and that a corrupted package, a failed
signature policy, an incompatible release, a missing update checker and
a caller who is not an administrator each leave the installed version in
place. The new specs are added to the lifecycle case inventory.

The add-ons page describes the setting, its schedule and its default.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(addons): name the library entry identifier id

The library entry of an add-on used addonId while the shared envelope
and the model entries use id. Release now has the field ID, written as
id, and every writer emits it: the index encoder, the package workflow
templates, the catalog generator and gallery/addons/index.yaml. The
digests in the index did not change.

DecodeIndex still reads addonId as a legacy alias for one release, so
an existing library and a library served by an older tool keep
working. An entry that holds both names is refused. The Apps UI reads
id and falls back to addonId.

Assisted-by: Claude Code:claude-sonnet-5-5 go node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(addons): use the id field in the auto-upgrade test release

The auto-upgrade test arrived after the identifier rename and still
named the release field AddonID, so the application tests did not
build.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(addons): annotate the auto-upgrade actor label for gosec

The actor label looks like a credential to gosec G101. It is a fixed name for upgrades the owner starts itself.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(schema): persist the per-chat add-on tool fields

The chat header persists addonTools and addonToolIds, but Conversation had no fields for them, so the server dropped them on save. Add both and update the generated swagger files by hand because swag is not installed here.

Assisted-by: Claude Code:claude-sonnet-5-5 go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): count the add-ons auto-upgrade setting in the settings groups

The setting sits in the Backends and galleries group, once. The page now lists 54 settings plus the three branding images, so the spec expects 57.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): pause the fake clock ahead of now in the clear-all spec

The fake clock runs from install, so pausing at the current time failed with 'Cannot fast-forward to the past' on a slow runner. The save itself is synchronous and was not lost.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): hold the Studio capabilities reply longer in the skeleton spec

800 ms was shorter than a slow runner needs to render the page, so the skeleton was gone before the check. The product shows it.

Assisted-by: Claude Code:claude-sonnet-5-5 node playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(i18n): add the French add-ons locale

Master added the fr locale after this branch forked. The add-ons unit test requires every locale to carry the add-ons namespace, so fr needs its own addons.json.

Assisted-by: Claude Code:claude-sonnet-5-5 node
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler-agentandEttore Di Giacinto authored and GitHub committed 2026-10-22 17:57:19 +02:00
1 parent 099d6c3dd0
commit e45dc5e9f7
964 files changed
+159560 -515
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

addons/studio/cmd/studio/main.go
+15
View File
@@ -0,0 +1,15 @@
// SPDX-License-Identifier: MIT
// Command studio is the executable of the Studio add-on. It serves no tools:
// the add-on works in the browser, through the SDK and the core API.
package main
import (
"os"
"github.com/mudler/LocalAI/addons/internal/idle"
)
func main() {
os.Exit(idle.Run(os.Args))
}
addons/studio/manifest.json
+25
View File
@@ -0,0 +1,25 @@
{
"schemaVersion": 1,
"id": "studio",
"version": "0.1.0",
"executable": "studio",
"os": "linux",
"arch": "amd64",
"host": { "min": 1, "max": 1 },
"api": { "min": 1, "max": 1 },
"protocol": { "min": 1, "max": 1 },
"dataFormat": { "min": 1, "max": 1 },
"capabilities": {
"ui": {
"entry": "ui/studio.js",
"css": ["ui/studio.css"],
"sdk": { "min": 2, "max": 2 },
"pages": ["studio"],
"navigation": ["studio"],
"mainPage": "studio",
"icon": "ui/icon.png",
"title": "Studio",
"description": "Make images, video, 3D models, speech, sound and transformed audio with the models you have installed, and keep what you make."
}
}
}
addons/studio/package/platforms.yaml
+38
View File
@@ -0,0 +1,38 @@
# Release platform matrix of the studio add-on package.
#
# A platform ships only while "enabled" is true. Enable a platform in the same
# change that shows its native check passing on a runner of that platform;
# until then it stays listed and disabled. The release workflow fails closed
# when no platform is enabled, and it never builds a platform that is not
# enabled here.
#
# runner: the GitHub runner label that builds and checks the platform
# natively. Required for an enabled platform.
# native_check: "executable-start": the packaged executable is extracted on
# the native runner and started. Required for an enabled platform.
# runtime: files besides the executable that the platform needs at run
# time, relative to the package root. They are taken from the
# runtime directory of the build and must exist.
schema: 1
addon: studio
executable: studio
manifest: addons/studio/manifest.json
# The directory whose content is laid at the package root (ui/studio.js ...).
ui: addons/studio/ui/dist
platforms:
- platform: linux/amd64
enabled: true
runner: ubuntu-24.04
native_check: executable-start
- platform: linux/arm64
enabled: true
runner: ubuntu-24.04-arm
native_check: executable-start
- platform: darwin/arm64
enabled: true
runner: macos-14
native_check: executable-start
- platform: darwin/amd64
enabled: true
runner: macos-15-intel
native_check: executable-start
addons/studio/ui/.gitignore
+2
View File
@@ -0,0 +1,2 @@
# Written by build.mjs; the package build makes it.
/dist/
addons/studio/ui/build.mjs
+15
View File
@@ -0,0 +1,15 @@
import { fileURLToPath } from 'node:url'
import { dirname, resolve } from 'node:path'
import { buildAddonUI } from '../../internal/uibuild/uibuild.mjs'
const here = dirname(fileURLToPath(import.meta.url))
export async function build() {
// The page code keeps the host module names it was written with; the compat
// directory answers them through the SDK (addons/internal/host-compat).
return buildAddonUI({ root: here, name: 'studio', rgb: [150, 52, 92], compat: resolve(here, '../../internal/host-compat/src') })
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
build().then((dir) => console.log(`built ${dir}`), (e) => { console.error(e); process.exit(1) })
}
addons/studio/ui/icon.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.6 KiB

addons/studio/ui/package.json
+8
View File
@@ -0,0 +1,8 @@
{
"name": "localai-studio-ui",
"private": true,
"version": "0.1.0",
"description": "Native UI of the Studio add-on: a standalone ES module that the host loads after its own build.",
"type": "module",
"scripts": { "build": "node build.mjs", "test": "node test/run.mjs" }
}
addons/studio/ui/src/addon.jsx
+27
View File
@@ -0,0 +1,27 @@
import { RouteProvider } from 'react-router-dom'
import { registerLocales } from '../../../internal/host-compat/src/i18n.js'
import Studio from './pages/Studio'
import en from './i18n/media.en.json'
import it from './i18n/media.it.json'
import es from './i18n/media.es.json'
import de from './i18n/media.de.json'
import zh from './i18n/media.zh-CN.json'
import id from './i18n/media.id.json'
import ko from './i18n/media.ko.json'
import pt from './i18n/media.pt-BR.json'
// The words of the pages are the host's own `media` locale files.
registerLocales({ media: { en, it, es, de, 'zh-CN': zh, id, ko, 'pt-BR': pt } })
export function createStudio() {
// The shell passes the page, the rest of the path (the tab) and the query.
function Component({ page, path, search }) {
if (page !== 'studio') return null
return (
<RouteProvider value={{ page, path: path || '', search: search || '' }}>
<Studio />
</RouteProvider>
)
}
return { Component, components: {}, chatActions: [], dispose() {} }
}
addons/studio/ui/src/components/AnimationOptions.jsx
+93
View File
@@ -0,0 +1,93 @@
// SPDX-License-Identifier: MIT
import { useEffect, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { fileToBase64 } from '../utils/api'
import Icon from './Icon'
function AnimationInput({ input, value, onChange }) {
const { t } = useTranslation('media')
const [error, setError] = useState('')
const selection = useRef(0)
useEffect(() => () => { selection.current++ }, [])
const id = `animation-${input.name}`
const data = value?.data || ''
const tooLong = input.type === 'text' && input.max_bytes && new TextEncoder().encode(data).length > input.max_bytes
const byteError = tooLong ? t('threed.animation.inputTooLong', 'Input exceeds the {{limit}} byte limit.', { limit: input.max_bytes }) : ''
return (
<div className="form-group">
<label className="form-label" htmlFor={id}>{input.label}</label>
{input.type === 'text' ? <textarea
id={id} className="input" rows={4} required={input.required} value={data}
ref={element => element?.setCustomValidity(byteError)}
aria-invalid={!!byteError}
onChange={event => onChange({ type: input.type, data: event.target.value })}
/> : <input
id={id} className="input" type="file" required={input.required && !data}
accept={{ image: 'image/*', video: 'video/*', mesh: '.glb,.gltf,.obj,.ply,.stl' }[input.type]}
onChange={async event => {
const current = ++selection.current
const file = event.target.files?.[0]
setError('')
onChange(null)
if (!file) return
try {
if (file.size > (input.max_bytes || 32 * 1024 * 1024)) throw new Error(t('threed.animation.fileTooLarge', 'The selected file is too large.'))
const encoded = await fileToBase64(file)
if (current === selection.current) onChange({ type: input.type, data: encoded })
} catch (err) {
if (current === selection.current) setError(err?.message || t('threed.animation.readFailed', 'Could not read the selected file.'))
}
}}
/>}
{(byteError || error) && <p role="alert" className="form-error">{byteError || error}</p>}
</div>
)
}
export default function AnimationOptions({ operation, inputs, onInputsChange, params, onParamsChange }) {
const { t } = useTranslation('media')
const [advanced, setAdvanced] = useState(false)
return (
<div className="stack">
{operation.inputs.map(input => (
<AnimationInput key={input.name} input={input} value={inputs[input.name]}
onChange={value => onInputsChange(previous => {
const next = { ...previous }
if (value) next[input.name] = value
else delete next[input.name]
return next
})} />
))}
<div className="form-grid-2col">
{operation.parameters.filter(parameter => !parameter.advanced || advanced).map(parameter => (
<div className="form-group" key={parameter.name}>
<label className="form-label" htmlFor={`animation-${parameter.name}`}>{parameter.label}</label>
{parameter.type === 'enum' ? <select
id={`animation-${parameter.name}`} className="input"
value={params[parameter.name] ?? parameter.default ?? ''}
onChange={event => onParamsChange({ ...params, [parameter.name]: event.target.value })}
>
{parameter.options.map(value => <option key={value} value={value}>{value}</option>)}
</select> : <input
id={`animation-${parameter.name}`}
className="input"
type={parameter.type === 'uint64' ? 'text' : 'number'}
inputMode={parameter.type === 'uint64' ? 'numeric' : undefined}
pattern={parameter.type === 'uint64' ? '[0-9]{1,20}' : undefined}
min={parameter.min ?? 0}
max={parameter.max || undefined}
step={parameter.type === 'integer' ? 1 : 'any'}
placeholder={parameter.default}
value={params[parameter.name] ?? ''}
onChange={event => onParamsChange({ ...params, [parameter.name]: event.target.value })}
/>}
</div>
))}
</div>
<button type="button" className={`collapsible-header ${advanced ? 'open' : ''}`} aria-expanded={advanced} onClick={() => setAdvanced(!advanced)}>
<Icon name="chevron-right" /> {t('threed.labels.advanced')}
</button>
{operation.output === 'skeleton_animation' && <p className="form-hint">{t('threed.animation.skeletonHint', 'Generates an animated skeleton, without a mesh or skin.')}</p>}
</div>
)
}
addons/studio/ui/src/components/AnimationViewer.jsx
+121
View File
@@ -0,0 +1,121 @@
// SPDX-License-Identifier: MIT
import { useEffect, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { animationPose, parseAnimationGlb } from '../utils/animationGlb'
export default function AnimationViewer({ blob }) {
const { t } = useTranslation('media')
const canvas = useRef(null)
const drag = useRef(null)
const [animation, setAnimation] = useState(null)
const [error, setError] = useState(null)
const [playing, setPlaying] = useState(true)
const [time, setTime] = useState(0)
const [yaw, setYaw] = useState(-0.6)
const [zoom, setZoom] = useState(1)
const [size, setSize] = useState([640, 480])
useEffect(() => {
let cancelled = false
setAnimation(null)
setError(null)
setTime(0)
blob.arrayBuffer().then(parseAnimationGlb).then(value => {
animationPose(value, 0)
if (!cancelled) { setAnimation(value); setPlaying(true) }
}).catch(err => { if (!cancelled) setError(err.message) })
return () => { cancelled = true }
}, [blob])
useEffect(() => {
if (!playing || !animation?.duration) return
let frame, previous
const tick = now => {
if (previous === undefined) previous = now
if (now - previous >= 1000 / 30) {
// Capture elapsed time before React defers the update; dropped frames
// must skip ahead on the timeline rather than slow down playback.
const elapsed = (now - previous) / 1000
setTime(value => (value + elapsed) % animation.duration)
previous = now
}
frame = requestAnimationFrame(tick)
}
frame = requestAnimationFrame(tick)
return () => cancelAnimationFrame(frame)
}, [playing, animation])
useEffect(() => {
if (!canvas.current) return
const observer = new ResizeObserver(([entry]) => setSize([Math.round(entry.contentRect.width), Math.round(entry.contentRect.height)]))
observer.observe(canvas.current)
return () => observer.disconnect()
}, [])
useEffect(() => {
if (!canvas.current) return
const context = canvas.current.getContext('2d')
if (!context) return
context.clearRect(0, 0, ...size)
if (!animation) return
try {
const positions = animationPose(animation, time)
const [width, height] = size
const root = positions[animation.order[0]]
const scale = Math.min(width, height) * 0.32 * zoom
const project = ([x, y, z]) => {
x -= root[0]; z -= root[2]
return [width/2 + (x*Math.cos(yaw)-z*Math.sin(yaw))*scale,
height*0.8 - (y + (x*Math.sin(yaw)+z*Math.cos(yaw))*0.2)*scale]
}
const styles = getComputedStyle(canvas.current)
context.clearRect(0, 0, width, height)
context.strokeStyle = styles.getPropertyValue('--color-border').trim() || '#465566'
context.lineWidth = 1
for (let i = -5; i <= 5; i++) {
for (const ends of [[[root[0]-5, 0, root[2]+i], [root[0]+5, 0, root[2]+i]], [[root[0]+i, 0, root[2]-5], [root[0]+i, 0, root[2]+5]]]) {
const a = project(ends[0]), b = project(ends[1])
context.beginPath(); context.moveTo(...a); context.lineTo(...b); context.stroke()
}
}
context.strokeStyle = styles.getPropertyValue('--color-primary').trim() || '#3f9a91'
context.fillStyle = context.strokeStyle
context.lineWidth = 3
positions.forEach((position, index) => {
const point = project(position)
const parent = animation.nodes[index].parent
if (parent >= 0) {
context.beginPath(); context.moveTo(...project(positions[parent])); context.lineTo(...point); context.stroke()
}
context.beginPath(); context.arc(...point, 3, 0, 2*Math.PI); context.fill()
})
} catch (err) { setError(err.message); setPlaying(false) }
}, [animation, time, yaw, zoom, size])
return (
<div className="stack animation-viewer" data-testid="animation-viewer">
{error && <p role="alert" className="form-error">{error}</p>}
<canvas
ref={canvas} width={size[0]} height={size[1]}
aria-label={t('threed.animation.preview', 'Animated skeleton preview')}
onPointerDown={event => { drag.current = event.clientX; event.currentTarget.setPointerCapture(event.pointerId) }}
onPointerMove={event => { if (drag.current !== null) { setYaw(value => value + (event.clientX-drag.current)*0.01); drag.current = event.clientX } }}
onPointerUp={() => { drag.current = null }}
onPointerCancel={() => { drag.current = null }}
/>
{animation && <div className="stack">
<div className="hstack">
<button type="button" className="btn btn-secondary" onClick={() => setPlaying(!playing)}>
{playing ? t('threed.animation.pause', 'Pause') : t('threed.animation.play', 'Play')}
</button>
<button type="button" className="btn btn-secondary" onClick={() => { setYaw(-0.6); setZoom(1) }}>{t('threed.animation.reset', 'Reset view')}</button>
<output data-testid="animation-time">{time.toFixed(2)} / {animation.duration.toFixed(2)} s</output>
</div>
<label className="form-label" htmlFor="animation-timeline">{t('threed.animation.timeline', 'Timeline')}</label>
<input id="animation-timeline" type="range" min="0" max={animation.duration} step="0.01" value={time} onChange={event => { setPlaying(false); setTime(Number(event.target.value)) }} />
<label className="form-label" htmlFor="animation-zoom">{t('threed.animation.zoom', 'Zoom')}</label>
<input id="animation-zoom" type="range" min="0.3" max="3" step="0.05" value={zoom} onChange={event => setZoom(Number(event.target.value))} />
</div>}
</div>
)
}
addons/studio/ui/src/components/GlbViewer.jsx
+667
View File
@@ -0,0 +1,667 @@
import { useEffect, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { parseGlb } from '../utils/glb'
import Icon from './Icon'
/* ── WebGL2 GLB viewer ──────────────────────────────────────────────────────
* Ported from the trellis2cpp demo server's hand-rolled viewer
* (localai-org/trellis2cpp server/web/index.html): quaternion trackball,
* metallic-roughness PBR with a procedural environment, ACES tonemapping,
* hidden-line wireframe. Kept dependency-free — the renderer is tuned for
* the dense unoriented dual-grid meshes TRELLIS.2 produces, which generic
* GLTF viewers shade poorly.
*
* Deltas from the demo: input is a parsed GLB (see utils/glb.js) instead of
* the demo's T2MESH stream; COLOR_0 arrives LINEAR (no gamma decode — the
* demo's pow(2.2) would double-darken it); vertex attributes upload as
* normalized integers straight from the GLB buffers; an optional UV-texture
* path covers the atlas-baked form; the base orientation drops the demo's
* Z-up -> Y-up turn because baked GLBs are already Y-up.
*/
const VS = `#version 300 es
layout(location=0) in vec3 pos;
layout(location=1) in vec3 nrm;
layout(location=2) in vec4 baseColor;
layout(location=3) in vec2 metalRough;
layout(location=4) in vec2 uv;
uniform mat4 mvp, viewModel;
out vec3 vN; out vec3 vV; out vec4 vCol; out vec2 vMR; out vec2 vUV;
void main() {
gl_Position = mvp * vec4(pos, 1.0);
vec4 vp = viewModel * vec4(pos, 1.0);
vV = -vp.xyz; // view-space: to-camera (camera at origin)
vN = mat3(viewModel) * nrm; // view-space normal (orbits with camera)
vCol = baseColor;
vMR = metalRough;
vUV = uv;
}`
// Lightweight metallic-roughness PBR in view space. The dual-grid mesh has
// unoriented winding (faithful to TRELLIS.2), so every normal is faced toward
// the camera. No IBL — a procedural environment + one studio key light;
// ACES tonemapping at the end.
const FS = `#version 300 es
precision highp float;
in vec3 vN; in vec3 vV; in vec4 vCol; in vec2 vMR; in vec2 vUV;
uniform int wire;
uniform int colorMode; // 0 = untextured grey, 1 = vertex PBR (linear COLOR_0), 2 = UV textures
uniform sampler2D baseColorTex;
uniform sampler2D mrTex;
uniform float uMetallicFactor, uRoughnessFactor;
out vec4 frag;
// procedural environment radiance in a direction (view space; +y = up)
vec3 envColor(vec3 d) {
vec3 sky = mix(vec3(0.32, 0.40, 0.55), vec3(0.72, 0.80, 0.98), clamp(d.y, 0.0, 1.0));
vec3 gnd = vec3(0.14, 0.13, 0.12);
return mix(gnd, sky, smoothstep(-0.30, 0.12, d.y));
}
void main() {
if (wire == 1) { frag = vec4(0.30, 0.64, 1.00, 1.0); return; }
vec3 N = normalize(vN), V = normalize(vV);
if (dot(N, V) < 0.0) N = -N; // face the camera (mesh is unoriented)
vec3 base = vec3(0.62, 0.66, 0.72);
float metal = 0.0, rough = 0.5, opacity = 1.0;
if (colorMode == 1) {
// COLOR_0 is stored linear in the GLB — use it directly.
base = clamp(vCol.rgb, 0.0, 1.0);
metal = clamp(vMR.x, 0.0, 1.0);
rough = clamp(vMR.y, 0.06, 1.0);
opacity = clamp(vCol.a, 0.0, 1.0);
} else if (colorMode == 2) {
vec4 bc = texture(baseColorTex, vUV);
base = pow(clamp(bc.rgb, 0.0, 1.0), vec3(2.2)); // baseColorTexture is sRGB
opacity = bc.a;
vec3 mr = texture(mrTex, vUV).rgb; // G=roughness, B=metallic
metal = clamp(mr.b * uMetallicFactor, 0.0, 1.0);
rough = clamp(mr.g * uRoughnessFactor, 0.06, 1.0);
}
if (opacity < 0.01) discard;
float nv = max(dot(N, V), 1e-3);
// Schlick Fresnel (grazing reflectance rises to 1 - roughness for metals).
vec3 F0 = mix(vec3(0.04), base, metal);
vec3 F = F0 + (max(vec3(1.0 - rough), F0) - F0) * pow(1.0 - nv, 5.0);
// diffuse: hemispheric environment irradiance (metals have no diffuse)
vec3 irr = envColor(N) * 0.55 + vec3(0.12);
vec3 diffuse = base * (1.0 - metal) * irr;
// specular: environment reflection, blurred toward a flat tint by roughness
vec3 refl = mix(envColor(reflect(-V, N)), vec3(0.34, 0.37, 0.44), rough * rough);
vec3 specular = refl * F;
// one crisp studio key light for a lively highlight
vec3 L = normalize(vec3(0.45, 0.70, 0.55)), H = normalize(L + V);
float shin = mix(8.0, 260.0, pow(1.0 - rough, 2.0));
float sp = pow(max(dot(N, H), 0.0), shin) * (shin + 2.0) / 6.2831853;
vec3 kc = vec3(1.00, 0.96, 0.88);
float ndl = max(dot(N, L), 0.0);
specular += kc * sp * F * ndl;
diffuse += base * (1.0 - metal) * kc * ndl * 0.28;
vec3 color = diffuse + specular;
color = (color * (2.51 * color + 0.03)) / (color * (2.43 * color + 0.59) + 0.14); // ACES
frag = vec4(pow(clamp(color, 0.0, 1.0), vec3(1.0/2.2)), opacity);
}`
/* trackball orientation (quaternion): each drag composes a small rotation
* about the screen axes onto the current orientation — no gimbal lock. */
const Q = {
axisAngle(x, y, z, a) { const h = a * 0.5, s = Math.sin(h); return [x * s, y * s, z * s, Math.cos(h)] },
mul(a, b) { // Hamilton product a·b (apply b, then a)
return [
a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1],
a[3] * b[1] - a[0] * b[2] + a[1] * b[3] + a[2] * b[0],
a[3] * b[2] + a[0] * b[1] - a[1] * b[0] + a[2] * b[3],
a[3] * b[3] - a[0] * b[0] - a[1] * b[1] - a[2] * b[2],
]
},
norm(q) { const n = Math.hypot(q[0], q[1], q[2], q[3]) || 1; return [q[0] / n, q[1] / n, q[2] / n, q[3] / n] },
toMat4(q) { // column-major rotation matrix
const [x, y, z, w] = q
const xx = x * x, yy = y * y, zz = z * z, xy = x * y, xz = x * z, yz = y * z, wx = w * x, wy = w * y, wz = w * z
return new Float32Array([
1 - 2 * (yy + zz), 2 * (xy + wz), 2 * (xz - wy), 0,
2 * (xy - wz), 1 - 2 * (xx + zz), 2 * (yz + wx), 0,
2 * (xz + wy), 2 * (yz - wx), 1 - 2 * (xx + yy), 0,
0, 0, 0, 1,
])
},
}
// GLBs are already Y-up (the baker swaps axes on export), so unlike the demo
// there is no Z-up correction here — just a gentle 3/4 default view.
const QBASE = Q.norm(Q.mul(Q.axisAngle(1, 0, 0, -0.30), Q.axisAngle(0, 1, 0, 0.55)))
const FRAME_INTERVAL_MS = 1000 / 30
/* minimal mat4 helpers (column-major) */
const M = {
mul(a, b) {
const o = new Float32Array(16)
for (let c = 0; c < 4; ++c) for (let r = 0; r < 4; ++r)
o[c * 4 + r] = a[r] * b[c * 4] + a[4 + r] * b[c * 4 + 1] + a[8 + r] * b[c * 4 + 2] + a[12 + r] * b[c * 4 + 3]
return o
},
persp(fov, asp, near, far) {
const f = 1 / Math.tan(fov / 2), o = new Float32Array(16)
o[0] = f / asp; o[5] = f
o[10] = (far + near) / (near - far); o[11] = -1
o[14] = 2 * far * near / (near - far)
return o
},
trans(x, y, z) {
const o = new Float32Array(16)
o[0] = o[5] = o[10] = o[15] = 1
o[12] = x; o[13] = y; o[14] = z
return o
},
scale(s) {
const o = new Float32Array(16)
o[0] = o[5] = o[10] = s; o[15] = 1
return o
},
}
function glType(gl, array) {
if (array instanceof Uint8Array) return gl.UNSIGNED_BYTE
if (array instanceof Uint16Array) return gl.UNSIGNED_SHORT
return gl.FLOAT
}
export function createGlbViewer(canvas, { onContextLost } = {}) {
const gl = canvas.getContext('webgl2', { antialias: true })
if (!gl) return null
// If the GPU context is lost (driver reset / out of memory), preventDefault
// keeps it recoverable and the page shows a notice instead of a dead canvas.
const contextLost = (e) => {
e.preventDefault()
if (onContextLost) onContextLost()
}
canvas.addEventListener('webglcontextlost', contextLost, false)
function shader(type, src) {
const s = gl.createShader(type)
gl.shaderSource(s, src); gl.compileShader(s)
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw new Error(gl.getShaderInfoLog(s))
return s
}
const prog = gl.createProgram()
gl.attachShader(prog, shader(gl.VERTEX_SHADER, VS))
gl.attachShader(prog, shader(gl.FRAGMENT_SHADER, FS))
gl.linkProgram(prog)
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) throw new Error(gl.getProgramInfoLog(prog))
const uMVP = gl.getUniformLocation(prog, 'mvp')
const uViewModel = gl.getUniformLocation(prog, 'viewModel')
const uWire = gl.getUniformLocation(prog, 'wire')
const uColorMode = gl.getUniformLocation(prog, 'colorMode')
const uBaseColorTex = gl.getUniformLocation(prog, 'baseColorTex')
const uMrTex = gl.getUniformLocation(prog, 'mrTex')
const uMetallicFactor = gl.getUniformLocation(prog, 'uMetallicFactor')
const uRoughnessFactor = gl.getUniformLocation(prog, 'uRoughnessFactor')
const vao = gl.createVertexArray()
const vbo = gl.createBuffer(), nbo = gl.createBuffer(), cbo = gl.createBuffer()
const mbo = gl.createBuffer(), ubo = gl.createBuffer(), ibo = gl.createBuffer()
const wireIbo = gl.createBuffer()
let nIndices = 0, nWire = 0
let colorMode = 0
let baseColorTexture = null, mrTexture = null
let metallicFactor = 1, roughnessFactor = 1
// fit-to-view: model = rotation * scale * translate(-center) keeps the demo's
// camera constants valid for any GLB extent (trellis meshes are ~unit cube).
let center = [0, 0, 0], fitScale = 1
let rot = QBASE.slice(), dist = 1.8, panX = 0, panY = 0
let wire = false, spin = true
let disposed = false
function makeTexture(bitmap) {
const tex = gl.createTexture()
gl.bindTexture(gl.TEXTURE_2D, tex)
// Matches the baker's sampler: linear filtering, clamp, no mipmaps.
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, bitmap)
gl.bindTexture(gl.TEXTURE_2D, null)
return tex
}
function dropTextures() {
if (baseColorTexture) { gl.deleteTexture(baseColorTexture); baseColorTexture = null }
if (mrTexture) { gl.deleteTexture(mrTexture); mrTexture = null }
}
async function setMesh(mesh) {
gl.bindVertexArray(vao)
gl.bindBuffer(gl.ARRAY_BUFFER, vbo)
gl.bufferData(gl.ARRAY_BUFFER, mesh.positions, gl.STATIC_DRAW)
gl.enableVertexAttribArray(0)
gl.vertexAttribPointer(0, 3, gl.FLOAT, false, 0, 0)
if (mesh.normals) {
gl.bindBuffer(gl.ARRAY_BUFFER, nbo)
gl.bufferData(gl.ARRAY_BUFFER, mesh.normals, gl.STATIC_DRAW)
gl.enableVertexAttribArray(1)
gl.vertexAttribPointer(1, 3, gl.FLOAT, false, 0, 0)
} else {
gl.disableVertexAttribArray(1)
gl.vertexAttrib3f(1, 0, 1, 0)
}
dropTextures()
colorMode = 0
if (mesh.color0) {
// Upload COLOR_0 / _METALLIC_ROUGHNESS as normalized integers straight
// from the GLB buffers — no CPU conversion of multi-million-vertex data.
gl.bindBuffer(gl.ARRAY_BUFFER, cbo)
gl.bufferData(gl.ARRAY_BUFFER, mesh.color0.array, gl.STATIC_DRAW)
gl.enableVertexAttribArray(2)
gl.vertexAttribPointer(2, mesh.color0.size, glType(gl, mesh.color0.array), mesh.color0.normalized, 0, 0)
if (mesh.metalRough) {
gl.bindBuffer(gl.ARRAY_BUFFER, mbo)
gl.bufferData(gl.ARRAY_BUFFER, mesh.metalRough.array, gl.STATIC_DRAW)
gl.enableVertexAttribArray(3)
gl.vertexAttribPointer(3, mesh.metalRough.size, glType(gl, mesh.metalRough.array), mesh.metalRough.normalized, 0, 0)
} else {
gl.disableVertexAttribArray(3)
gl.vertexAttrib2f(3, 0, 0.6)
}
gl.disableVertexAttribArray(4)
colorMode = 1
} else if (mesh.uv && mesh.baseColorPng) {
gl.bindBuffer(gl.ARRAY_BUFFER, ubo)
gl.bufferData(gl.ARRAY_BUFFER, mesh.uv, gl.STATIC_DRAW)
gl.enableVertexAttribArray(4)
gl.vertexAttribPointer(4, 2, gl.FLOAT, false, 0, 0)
gl.disableVertexAttribArray(2)
gl.disableVertexAttribArray(3)
const bitmaps = await Promise.all([
createImageBitmap(new Blob([mesh.baseColorPng], { type: 'image/png' })),
mesh.metalRoughPng ? createImageBitmap(new Blob([mesh.metalRoughPng], { type: 'image/png' })) : null,
])
if (disposed) return
gl.bindVertexArray(vao)
baseColorTexture = makeTexture(bitmaps[0])
mrTexture = bitmaps[1] ? makeTexture(bitmaps[1]) : makeTexture(bitmaps[0])
metallicFactor = mesh.material.metallicFactor
roughnessFactor = mesh.material.roughnessFactor
colorMode = 2
} else {
gl.disableVertexAttribArray(2)
gl.disableVertexAttribArray(3)
gl.disableVertexAttribArray(4)
}
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, ibo)
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, mesh.indices, gl.STATIC_DRAW)
nIndices = mesh.indices.length
// wireframe index buffer: 3 edges per triangle, bounded to WIRE_BUDGET —
// a full wireframe of a multi-million-triangle mesh would OOM the GPU and
// lose the context (sub-pixel lines also fill in as a solid mass).
const WIRE_BUDGET = 12_000_000 // ~6M segments, ~48 MB
const nTri = nIndices / 3
const wireStride = Math.max(1, Math.ceil(nTri * 6 / WIRE_BUDGET))
const wireIdx = new Uint32Array(Math.ceil(nTri / wireStride) * 6)
let o = 0
const idx = mesh.indices
for (let t = 0; t < nIndices; t += 3 * wireStride) {
wireIdx[o++] = idx[t]; wireIdx[o++] = idx[t + 1]
wireIdx[o++] = idx[t + 1]; wireIdx[o++] = idx[t + 2]
wireIdx[o++] = idx[t + 2]; wireIdx[o++] = idx[t]
}
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, wireIbo)
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, wireIdx.subarray(0, o), gl.STATIC_DRAW)
nWire = o
gl.bindVertexArray(null)
center = [
(mesh.bboxMin[0] + mesh.bboxMax[0]) / 2,
(mesh.bboxMin[1] + mesh.bboxMax[1]) / 2,
(mesh.bboxMin[2] + mesh.bboxMax[2]) / 2,
]
const radius = Math.hypot(
mesh.bboxMax[0] - mesh.bboxMin[0],
mesh.bboxMax[1] - mesh.bboxMin[1],
mesh.bboxMax[2] - mesh.bboxMin[2],
) / 2 || 1
fitScale = 0.866 / radius
resetView()
}
function clear() {
nIndices = 0
nWire = 0
dropTextures()
requestRender()
}
function resetView() {
rot = QBASE.slice(); dist = 1.8; panX = panY = 0
requestRender()
}
/* input */
let dragging = false, panning = false, lx = 0, ly = 0
let pinchDistance = 0, pinchX = 0, pinchY = 0
const pointers = new Map()
const pointerPair = () => Array.from(pointers.values()).slice(0, 2)
const beginPinch = () => {
const [a, b] = pointerPair()
if (!a || !b) return
pinchDistance = Math.hypot(b.x - a.x, b.y - a.y)
pinchX = (a.x + b.x) / 2
pinchY = (a.y + b.y) / 2
}
const stopSpin = () => {
spin = false
requestRender()
if (onSpinChange) onSpinChange(false)
}
const onPointerDown = (e) => {
e.preventDefault()
canvas.setPointerCapture(e.pointerId)
pointers.set(e.pointerId, { x: e.clientX, y: e.clientY })
if (pointers.size === 1) {
dragging = true
panning = e.button === 2 || e.shiftKey
lx = e.clientX; ly = e.clientY
} else {
dragging = false
beginPinch()
stopSpin()
}
}
const onPointerUp = (e) => {
pointers.delete(e.pointerId)
if (canvas.hasPointerCapture(e.pointerId)) canvas.releasePointerCapture(e.pointerId)
pinchDistance = 0
if (pointers.size === 1) {
const remaining = pointers.values().next().value
dragging = true
panning = false
lx = remaining.x; ly = remaining.y
} else {
dragging = false
}
}
const onPointerMove = (e) => {
if (!pointers.has(e.pointerId)) return
pointers.set(e.pointerId, { x: e.clientX, y: e.clientY })
if (pointers.size >= 2) {
const [a, b] = pointerPair()
const nextDistance = Math.hypot(b.x - a.x, b.y - a.y)
const nextX = (a.x + b.x) / 2
const nextY = (a.y + b.y) / 2
if (pinchDistance > 0 && nextDistance > 0) {
dist *= pinchDistance / nextDistance
dist = Math.max(0.3, Math.min(8, dist))
panX += (nextX - pinchX) * 0.0015 * dist
panY -= (nextY - pinchY) * 0.0015 * dist
}
pinchDistance = nextDistance
pinchX = nextX
pinchY = nextY
requestRender()
return
}
if (!dragging) return
const dx = e.clientX - lx, dy = e.clientY - ly
lx = e.clientX; ly = e.clientY
if (panning) {
panX += dx * 0.0015 * dist; panY -= dy * 0.0015 * dist
} else {
// Compose screen-axis turns onto the current orientation (fixed camera
// frame), so rotation stays screen-relative and never locks up.
const k = 0.008
rot = Q.norm(Q.mul(Q.axisAngle(1, 0, 0, dy * k), Q.mul(Q.axisAngle(0, 1, 0, dx * k), rot)))
stopSpin()
}
requestRender()
}
const onContextMenu = (e) => e.preventDefault()
const onWheel = (e) => {
e.preventDefault()
dist *= Math.exp(e.deltaY * 0.001)
dist = Math.max(0.3, Math.min(8, dist))
requestRender()
}
const onDblClick = () => resetView()
let onSpinChange = null
canvas.addEventListener('pointerdown', onPointerDown)
canvas.addEventListener('pointerup', onPointerUp)
canvas.addEventListener('pointercancel', onPointerUp)
canvas.addEventListener('pointermove', onPointerMove)
canvas.addEventListener('contextmenu', onContextMenu)
canvas.addEventListener('wheel', onWheel, { passive: false })
canvas.addEventListener('dblclick', onDblClick)
gl.enable(gl.DEPTH_TEST)
gl.enable(gl.BLEND)
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA)
gl.clearColor(0.063, 0.078, 0.094, 1)
let rafId = 0
let lastDraw = 0
let dirty = true
function requestRender() {
dirty = true
if (!disposed && !rafId) rafId = requestAnimationFrame(frame)
}
function frame(now) {
rafId = 0
if (disposed) return
// requestAnimationFrame follows the display refresh rate, which can be
// 120-240 Hz. Skip expensive mesh draws until the 30 FPS budget is due.
if (spin && lastDraw && now - lastDraw < FRAME_INTERVAL_MS) {
rafId = requestAnimationFrame(frame)
return
}
if (!spin && !dirty) return
const dt = lastDraw ? Math.min((now - lastDraw) / 1000, 0.1) : 0
lastDraw = now
dirty = false
// auto-rotate: a slow turn about the screen-vertical axis (turntable feel)
if (spin) rot = Q.norm(Q.mul(Q.axisAngle(0, 1, 0, dt * 0.4), rot))
const w = canvas.clientWidth, h = canvas.clientHeight
if (w > 0 && h > 0 && (canvas.width !== w * devicePixelRatio || canvas.height !== h * devicePixelRatio)) {
canvas.width = w * devicePixelRatio; canvas.height = h * devicePixelRatio
}
gl.viewport(0, 0, canvas.width, canvas.height)
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT)
if (nIndices) {
const rotation = Q.toMat4(rot)
const model = M.mul(rotation, M.mul(M.scale(fitScale), M.trans(-center[0], -center[1], -center[2])))
const view = M.trans(panX, panY, -dist)
const viewModel = M.mul(view, model)
const proj = M.persp(0.9, (w || 1) / (h || 1), 0.05, 100)
const mvp = M.mul(proj, viewModel)
gl.useProgram(prog)
gl.uniformMatrix4fv(uMVP, false, mvp)
gl.uniformMatrix4fv(uViewModel, false, viewModel)
gl.uniform1f(uMetallicFactor, metallicFactor)
gl.uniform1f(uRoughnessFactor, roughnessFactor)
if (colorMode === 2) {
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, baseColorTexture)
gl.uniform1i(uBaseColorTex, 0)
gl.activeTexture(gl.TEXTURE1)
gl.bindTexture(gl.TEXTURE_2D, mrTexture)
gl.uniform1i(uMrTex, 1)
}
gl.bindVertexArray(vao)
if (wire) {
// hidden-line wireframe: a depth-only prepass (pushed back a hair via
// polygon offset) occludes back-facing edges, so the front surface's
// edges show instead of a see-through blob.
gl.enable(gl.POLYGON_OFFSET_FILL)
gl.polygonOffset(1.0, 1.0)
gl.colorMask(false, false, false, false)
gl.uniform1i(uWire, 0)
gl.uniform1i(uColorMode, 0)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, ibo)
gl.drawElements(gl.TRIANGLES, nIndices, gl.UNSIGNED_INT, 0)
gl.colorMask(true, true, true, true)
gl.disable(gl.POLYGON_OFFSET_FILL)
gl.uniform1i(uWire, 1)
// Front-surface edges sit at ~equal depth to the offset-back fill, so
// LEQUAL lets them pass while occluded edges (greater depth) fail.
gl.depthFunc(gl.LEQUAL)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, wireIbo)
gl.drawElements(gl.LINES, nWire, gl.UNSIGNED_INT, 0)
gl.depthFunc(gl.LESS)
} else {
gl.uniform1i(uWire, 0)
gl.uniform1i(uColorMode, colorMode)
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, ibo)
gl.drawElements(gl.TRIANGLES, nIndices, gl.UNSIGNED_INT, 0)
}
gl.bindVertexArray(null)
}
// A still model is complete until input, resize, or a control invalidates
// it. Spinning models keep scheduling frames, subject to the cap above.
if (spin) rafId = requestAnimationFrame(frame)
}
const resizeObserver = typeof ResizeObserver === 'undefined'
? null
: new ResizeObserver(requestRender)
resizeObserver?.observe(canvas)
requestRender()
function dispose() {
disposed = true
cancelAnimationFrame(rafId)
resizeObserver?.disconnect()
canvas.removeEventListener('pointerdown', onPointerDown)
canvas.removeEventListener('pointerup', onPointerUp)
canvas.removeEventListener('pointercancel', onPointerUp)
canvas.removeEventListener('pointermove', onPointerMove)
canvas.removeEventListener('contextmenu', onContextMenu)
canvas.removeEventListener('wheel', onWheel)
canvas.removeEventListener('dblclick', onDblClick)
canvas.removeEventListener('webglcontextlost', contextLost)
dropTextures()
gl.deleteBuffer(vbo); gl.deleteBuffer(nbo); gl.deleteBuffer(cbo)
gl.deleteBuffer(mbo); gl.deleteBuffer(ubo); gl.deleteBuffer(ibo)
gl.deleteBuffer(wireIbo)
gl.deleteVertexArray(vao)
gl.deleteProgram(prog)
gl.getExtension('WEBGL_lose_context')?.loseContext()
}
return {
setMesh,
clear,
dispose,
resetView,
setWire(v) { wire = v; requestRender() },
setSpin(v) { spin = v; requestRender() },
onSpinChanged(fn) { onSpinChange = fn },
}
}
export default function GlbViewer({ blob }) {
const { t } = useTranslation('media')
const canvasRef = useRef(null)
const viewerRef = useRef(null)
const [glError, setGlError] = useState(null) // 'no-webgl2' | 'context-lost' | parse error text
const [stats, setStats] = useState(null)
const [wire, setWire] = useState(false)
const [spin, setSpin] = useState(true)
useEffect(() => { // GL lifecycle — once per mount
let viewer = null
try {
viewer = createGlbViewer(canvasRef.current, { onContextLost: () => setGlError('context-lost') })
} catch {
viewer = null
}
if (!viewer) {
setGlError('no-webgl2')
return undefined
}
viewer.onSpinChanged(setSpin)
viewerRef.current = viewer
return () => {
viewerRef.current = null
viewer.dispose()
}
}, [])
useEffect(() => { // (re)load when the blob changes
if (!blob) {
viewerRef.current?.clear()
setStats(null)
return undefined
}
let cancelled = false
;(async () => {
try {
// Parse before touching GL: stats and parse errors surface even when
// WebGL2 is unavailable (e.g. headless CI), and the download button
// keeps working either way.
const mesh = parseGlb(await blob.arrayBuffer())
if (cancelled) return
setStats({
nVerts: mesh.nVerts,
nTris: mesh.nTris,
pbr: !!(mesh.color0 || mesh.baseColorPng),
})
if (viewerRef.current) await viewerRef.current.setMesh(mesh)
} catch (err) {
if (!cancelled) setGlError(err.message)
}
})()
return () => { cancelled = true }
}, [blob])
const toggleWire = () => {
const next = !wire
setWire(next)
viewerRef.current?.setWire(next)
}
const toggleSpin = () => {
const next = !spin
setSpin(next)
viewerRef.current?.setSpin(next)
}
return (
<div className="glb-viewer" style={{ display: 'flex', flexDirection: 'column', gap: 'var(--spacing-sm)', width: '100%' }}>
<canvas
ref={canvasRef}
style={{ width: '100%', aspectRatio: '4 / 3', borderRadius: 'var(--radius-md)', background: '#101418', touchAction: 'none' }}
data-testid="glb-canvas"
/>
<div style={{ display: 'flex', alignItems: 'center', gap: 'var(--spacing-sm)', flexWrap: 'wrap' }}>
<button type="button" className={`btn btn-sm ${wire ? 'btn-primary' : 'btn-secondary'}`} onClick={toggleWire}>
<Icon name="square-dashed" /> {t('threed.viewer.wireframe')}
</button>
<button type="button" className={`btn btn-sm ${spin ? 'btn-primary' : 'btn-secondary'}`} onClick={toggleSpin}>
<Icon name="refresh" /> {t('threed.viewer.autoRotate')}
</button>
{stats && (
<span style={{ color: 'var(--color-text-muted)', fontSize: '0.85em' }} data-testid="glb-stats">
{t('threed.viewer.stats', { verts: stats.nVerts.toLocaleString(), tris: stats.nTris.toLocaleString() })}
{stats.pbr ? ' · PBR' : ''}
</span>
)}
</div>
{glError === 'no-webgl2' && <p style={{ color: 'var(--color-text-muted)' }}>{t('threed.viewer.noWebgl')}</p>}
{glError === 'context-lost' && <p style={{ color: 'var(--color-text-muted)' }}>{t('threed.viewer.contextLost')}</p>}
{glError && glError !== 'no-webgl2' && glError !== 'context-lost' && (
<p style={{ color: 'var(--color-error)' }}>{glError}</p>
)}
<p style={{ color: 'var(--color-text-muted)', fontSize: '0.8em', margin: 0 }}>{t('threed.viewer.hint')}</p>
</div>
)
}
addons/studio/ui/src/components/RequestPanel.jsx
+54
View File
@@ -0,0 +1,54 @@
import { useState } from 'react'
import { useTranslation } from 'react-i18next'
import { copyToClipboard } from '../utils/clipboard'
import { apiUrl } from '../utils/basePath'
// The request the form just built.
//
// LocalAI is an API-first product and Studio is the best place in the app to
// teach its own endpoints: the form stops being a black box, and a result worth
// keeping can be reproduced from a shell without reverse-engineering which
// fields the page sent.
//
// Rendered only once there is something to show. A panel describing a request
// nobody has made yet is a tutorial, not a record.
export default function RequestPanel({ endpoint, body, method = 'POST' }) {
const { t } = useTranslation('media')
const [copied, setCopied] = useState(false)
if (!endpoint || !body) return null
const json = JSON.stringify(body, null, 2)
const curl = [
`curl -X ${method} ${window.location.origin}${apiUrl(endpoint)} \\`,
` -H 'Content-Type: application/json' \\`,
` -d '${JSON.stringify(body)}'`,
].join('\n')
const onCopy = async () => {
const ok = await copyToClipboard(curl)
if (!ok) return
setCopied(true)
setTimeout(() => setCopied(false), 2000)
}
return (
<section className="request-panel">
<div className="request-panel__head">
<span className="request-panel__title">{t('request.heading')}</span>
<button type="button" className="request-panel__copy" onClick={onCopy}>
{copied ? t('request.copied') : t('request.copyCurl')}
</button>
</div>
<pre className="request-panel__code">
<code>
<span className="request-panel__method">{method}</span>{' '}
<span className="request-panel__endpoint">{endpoint}</span>
{'\n'}
{json}
</code>
</pre>
</section>
)
}
addons/studio/ui/src/components/audio/Spectrogram.jsx
+105
View File
@@ -0,0 +1,105 @@
import { useEffect, useRef } from 'react'
import useSpectrogram from '../../hooks/useSpectrogram'
// Spectrogram — canvas heatmap of a clip's magnitude STFT (time × frequency).
// Time runs left→right, frequency low→high bottom→top, brighter = more energy.
// Used on the AudioTransform page to show input next to output so the user can
// see which bands the model attenuates (dark gaps that were bright in the
// input). Mirrors WaveformPlayer's canvas/label/overlay structure.
export default function Spectrogram({ src, label, height = 140, testId }) {
const canvasRef = useRef(null)
const { spectrogram, frames, bins, maxFreq, duration, error, loading } = useSpectrogram(src)
useEffect(() => {
const canvas = canvasRef.current
if (!canvas) return
const dpr = window.devicePixelRatio || 1
const cssW = canvas.clientWidth
const cssH = height
canvas.width = Math.floor(cssW * dpr)
canvas.height = Math.floor(cssH * dpr)
const ctx = canvas.getContext('2d')
ctx.setTransform(dpr, 0, 0, dpr, 0, 0)
ctx.clearRect(0, 0, cssW, cssH)
if (!spectrogram || !frames || !bins) return
// Paint at native (frames × bins) resolution into an offscreen canvas,
// then let drawImage smooth-scale it up — far cheaper than filling
// cssW×cssH rects, and the GPU handles the interpolation.
const img = ctx.createImageData(frames, bins)
for (let f = 0; f < frames; f++) {
for (let b = 0; b < bins; b++) {
const [r, g, bl] = magma(spectrogram[f * bins + b])
// Flip the frequency axis: image row 0 is the top = highest freq.
const o = ((bins - 1 - b) * frames + f) * 4
img.data[o] = r
img.data[o + 1] = g
img.data[o + 2] = bl
img.data[o + 3] = 255
}
}
const off = document.createElement('canvas')
off.width = frames
off.height = bins
off.getContext('2d').putImageData(img, 0, 0)
ctx.imageSmoothingEnabled = true
ctx.drawImage(off, 0, 0, cssW, cssH)
}, [spectrogram, frames, bins, height])
if (!src) return null
return (
<div className="audio-spectrogram">
{label && <div className="audio-spectrogram__label">{label}</div>}
<div className="audio-spectrogram__canvas-wrap" style={{ height }}>
{error ? (
<div className="audio-spectrogram__error">{error}</div>
) : (
<canvas ref={canvasRef} data-testid={testId} style={{ width: '100%', height: '100%' }} />
)}
{maxFreq > 0 && !error && (
<>
<span className="audio-spectrogram__axis audio-spectrogram__axis--top">{fmtHz(maxFreq)}</span>
<span className="audio-spectrogram__axis audio-spectrogram__axis--bottom">0 Hz</span>
</>
)}
{duration > 0 && !error && (
<span className="audio-spectrogram__duration">{duration.toFixed(1)}s</span>
)}
{loading && !error && <div className="audio-spectrogram__loading">Analysing…</div>}
</div>
</div>
)
}
function fmtHz(hz) {
if (hz >= 1000) return `${(hz / 1000).toFixed(hz % 1000 === 0 ? 0 : 1)} kHz`
return `${Math.round(hz)} Hz`
}
// magma — compact perceptual colormap (black→purple→orange→white) sampled at 8
// control points and linearly interpolated. Perceptually uniform maps read
// far better for spectral magnitude than a raw hue ramp. v is clamped to [0,1].
const MAGMA = [
[0, 0, 4],
[40, 11, 84],
[101, 21, 110],
[159, 42, 99],
[212, 72, 66],
[245, 125, 21],
[250, 193, 39],
[252, 253, 191],
]
function magma(v) {
const t = v <= 0 ? 0 : v >= 1 ? 1 : v
const x = t * (MAGMA.length - 1)
const i = Math.floor(x)
const frac = x - i
const a = MAGMA[i]
const b = MAGMA[Math.min(i + 1, MAGMA.length - 1)]
return [
Math.round(a[0] + (b[0] - a[0]) * frac),
Math.round(a[1] + (b[1] - a[1]) * frac),
Math.round(a[2] + (b[2] - a[2]) * frac),
]
}
addons/studio/ui/src/components/studio/HandoffNote.jsx
+30
View File
@@ -0,0 +1,30 @@
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
// A workspace opened from the Studio front page with a result as its starting
// point says so, and says plainly when that result could not be loaded. It
// renders nothing when the page was opened any other way.
//
// source what useHandoffSource() returned
// handoff what useStudioHandoff() returned
// onClear drop the source and keep the rest of the form
export default function HandoffNote({ source, handoff, onClear, wanted = true }) {
const { t } = useTranslation('media')
if (!handoff.from || !wanted || source.status === 'none') return null
const name = source.item?.title || t('studio.handoff.aResult')
const failed = source.status === 'missing' || source.status === 'error'
return (
<div className={`studio-handoff${failed ? ' studio-handoff--failed' : ''}`} role={failed ? 'alert' : 'status'} data-testid="studio-handoff" data-status={source.status}>
<Icon name={failed ? 'alert-circle' : 'link'} />
<span className="studio-handoff__text">
{source.status === 'loading' && t('studio.handoff.loading', { name })}
{source.status === 'ready' && t('studio.handoff.ready', { name })}
{source.status === 'missing' && t('studio.handoff.missing')}
{source.status === 'error' && t('studio.handoff.error', { name })}
</span>
{onClear && source.status === 'ready' && (
<button type="button" className="dk-btn dk-btn--ghost dk-btn--sm" onClick={onClear}>{t('studio.handoff.remove')}</button>
)}
</div>
)
}
addons/studio/ui/src/components/studio/InstallNote.jsx
+128
View File
@@ -0,0 +1,128 @@
import { useEffect, useMemo, useState } from 'react'
// eslint-disable-next-line no-unused-vars
import { Link, useOutletContext } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
import { modelsApi } from '../../utils/api'
import { useModelSuggestion } from '../../hooks/useModelSuggestion'
import { useOperations } from '../../hooks/useOperations'
import { useResources } from '../../hooks/useResources'
import { hostMemory } from '../home/memory'
import { gbLabel, cssVars } from '../../utils/modelLedger'
// What a missing model needs, shown only when the person picks that type (or a
// hand-off needs it). A model, a size, the memory it needs, the memory that is
// free, and one Install button. Nothing starts until they press it, and the
// words they typed stay where they are: this note sits inside the composer and
// the composer never unmounts while a model installs.
//
// typeLabel the type's display name, "Video"
// onChanged called while an install runs so the caller can re-read which
// models are installed; the note goes away once one is
export default function InstallNote({ type, typeLabel, onChanged, className = '' }) {
const { t } = useTranslation('media')
const outlet = useOutletContext()
const addToast = outlet?.addToast
const { state, suggestion } = useModelSuggestion(type, true)
const { operations } = useOperations()
const { resources } = useResources(15000)
const [started, setStarted] = useState('')
const [failure, setFailure] = useState('')
const name = suggestion?.name
const op = useMemo(
() => name && operations.find(o => o.name === name && !o.isBackend && !o.isDeletion),
[operations, name],
)
const busy = !!op || (!!started && started === name)
// While the install runs, ask again which models exist. The server adds the
// model when the download ends, and this note unmounts when it does.
useEffect(() => {
if (!busy) return undefined
const id = setInterval(() => onChanged?.(), 3000)
return () => clearInterval(id)
}, [busy, onChanged])
const memory = hostMemory(resources)
const free = memory ? Math.max(0, memory.total - memory.used) : 0
const install = async () => {
setFailure('')
setStarted(name)
try {
await modelsApi.install(name)
addToast?.(t('studio.install.started', { model: name }), 'success')
} catch (err) {
setStarted('')
setFailure(err.message)
}
}
if (state === 'loading' || state === 'idle') {
return (
<div className={`studio-note ${className}`.trim()} data-testid="studio-install-note" data-state="loading" aria-busy="true">
<h3>{t('studio.install.title', { type: typeLabel })}</h3>
<div className="dk-skeleton dk-skeleton--line studio-note__skeleton" />
</div>
)
}
if (state !== 'ready') {
return (
<div className={`studio-note ${className}`.trim()} data-testid="studio-install-note" data-state={state}>
<h3>{t('studio.install.title', { type: typeLabel })}</h3>
<p>{state === 'error' ? t('studio.install.galleryDown') : t('studio.install.noneListed', { type: typeLabel })}</p>
<div className="studio-note__row">
<Link className="dk-btn dk-btn--secondary dk-btn--sm" to="/app/models">{t('studio.install.browse')}</Link>
</div>
</div>
)
}
const size = suggestion.sizeBytes > 0 ? gbLabel(suggestion.sizeBytes) : (suggestion.sizeDisplay || null)
const need = suggestion.vramBytes > 0 ? gbLabel(suggestion.vramBytes) : (suggestion.vramDisplay || null)
const progress = op && op.progress > 0 ? Math.round(op.progress) : 0
return (
<div className={`studio-note ${className}`.trim()} data-testid="studio-install-note" data-state={busy ? 'installing' : 'ready'}>
<h3>{t('studio.install.title', { type: typeLabel })}</h3>
<p>
<code className="studio-note__model">{name}</code>
{' '}
{size ? t('studio.install.size', { size }) : t('studio.install.sizeUnknown')}
{' '}
{need ? t('studio.install.need', { need }) : t('studio.install.needUnknown')}
{memory && ' '}
{memory && t('studio.install.free', { free: gbLabel(free) })}
{' '}
{t('studio.install.nothingStarts')}
</p>
<div className="studio-note__row">
{busy ? (
<div className="studio-note__progress" role="status">
<Icon name="spinner" spin />
<span>{t('studio.install.installing', { model: name })}{progress > 0 ? ` ${progress}%` : ''}</span>
{progress > 0 && (
<span
className="dk-meter studio-note__meter"
role="progressbar"
aria-label={t('studio.install.installing', { model: name })}
aria-valuenow={progress}
aria-valuemin={0}
aria-valuemax={100}
>
<span className="dk-meter-seg" style={cssVars({ '--dk-w': `${progress}%` })} />
</span>
)}
</div>
) : (
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={install} data-testid="studio-install">
<Icon name="download" /> {t('studio.install.button', { model: name })}
</button>
)}
{failure && <span className="studio-note__error" role="alert">{t('studio.install.failed', { message: failure })}</span>}
</div>
</div>
)
}
addons/studio/ui/src/components/studio/LineageView.jsx
+389
View File
@@ -0,0 +1,389 @@
import { useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
// eslint-disable-next-line no-unused-vars
import UpscaleComposer from './UpscaleComposer'
import Icon from '../Icon'
// eslint-disable-next-line no-unused-vars
import InstallNote from './InstallNote'
// eslint-disable-next-line no-unused-vars
import StarButton from './StarButton'
// eslint-disable-next-line no-unused-vars
import WorkThumb from './WorkThumb'
import { relativeTime } from '../../utils/format'
import { cssVars } from '../../utils/modelLedger'
import {
NEXT_STEPS, TYPE_ICON, TYPE_INFO, canTake, handoffPath, layoutLineage, neighbour, projectOf, suggestNext,
} from '../../utils/studioWork'
// A prompt can be a paragraph; a title is one line of it.
function clip(text, max) {
const s = String(text || '').trim()
return s.length > max ? `${s.slice(0, max).trimEnd()}…` : s
}
function isTyping(el) {
if (!el) return false
const tag = el.tagName
return tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT' || el.isContentEditable
}
// A result, or a stack of results that came from each other, as a board: what it
// was made from on the left, what came out of it on the right, the path through
// the selection drawn heavier, and one dashed step suggested from the models that
// are installed.
//
// "Run as a new take" and "Branch from here" open the workspace with the source
// and the words filled in. Upscale instead runs in the inline composer. A step the destination
// workspace cannot start from yet is listed but disabled, with the reason.
export default function LineageView({
items, workId, modalities, upscalers = [], onAddImage, defaultModel, onClose, onToggleFavourite, onHandoff, onModelsChanged,
}) {
const { t } = useTranslation('media')
const members = useMemo(() => projectOf(items, workId), [items, workId])
const [selectedId, setSelectedId] = useState(workId)
const [expanded, setExpanded] = useState(false)
const [draft, setDraft] = useState(null)
const boardRef = useRef(null)
// The set of types that have a model, and the first model of each.
const installed = useMemo(() => new Set(modalities.filter(m => m.installed.length > 0).map(m => m.key)), [modalities])
const label = (key) => t(`studio.tabs.${key}`)
const selected = members?.find(m => m.id === selectedId) || members?.[members.length - 1] || null
const suggestion = selected ? suggestNext(selected.type, installed) : null
const shownStep = draft ? { to: draft.to, edge: draft.edge } : suggestion
const ghost = selected && shownStep
? { id: `ghost:${selected.id}`, from: `res:${selected.id}`, to: shownStep.to, edge: shownStep.edge, draft: !!draft }
: null
const layout = useMemo(
() => (members ? layoutLineage(members, { selectedId: selected?.id, ghost }) : null),
// ghost is derived from the three values below
// eslint-disable-next-line react-hooks/exhaustive-deps
[members, selected?.id, ghost?.to, ghost?.edge, ghost?.draft],
)
const startDraft = (step) => {
if (!step || !step.supported) return
const text = step.edge === 'variation' ? (selected.title || '') : t(`studio.steps.${step.edge}.template`, { defaultValue: '' })
setDraft({ to: step.to, edge: step.edge, text })
setExpanded(true)
}
const take = () => {
const ok = canTake(selected)
if (!ok.ok) return
const textLike = TYPE_INFO[selected.type].input === 'text'
onHandoff(handoffPath(selected.type, {
prompt: textLike ? selected.title : '',
model: selected.model,
size: selected.params?.size || '',
from: selected.id,
edge: 'take',
}))
}
const openDraft = () => {
if (!draft) return
const textLike = TYPE_INFO[draft.to].input === 'text'
const ids = modalities.find(m => m.key === draft.to)?.installed || []
onHandoff(handoffPath(draft.to, {
prompt: textLike ? draft.text : '',
model: defaultModel(draft.to, ids),
from: selected.id,
edge: draft.edge,
}))
}
// Keys: arrows walk the board, B branches, Esc closes the draft, then the
// dock, then the view.
const keys = useRef({})
keys.current = { layout, selected, draft, expanded, suggestion, startDraft }
// A layout effect, so the keys work from the first painted frame.
useLayoutEffect(() => {
const onKey = (e) => {
const s = keys.current
if (e.key === 'Escape') {
if (s.draft) setDraft(null)
else if (s.expanded) setExpanded(false)
else onClose()
return
}
if (isTyping(e.target) || e.ctrlKey || e.metaKey || e.altKey || !s.layout) return
const dir = { ArrowLeft: 'left', ArrowRight: 'right', ArrowUp: 'up', ArrowDown: 'down' }[e.key]
if (dir) {
e.preventDefault()
const next = neighbour(s.layout, s.selected?.id, dir)
if (next) { setSelectedId(next); setDraft(null) }
} else if (e.key === 'b' || e.key === 'B') {
const step = s.suggestion || (NEXT_STEPS[s.selected?.type] || []).find(x => x.supported)
if (step) { e.preventDefault(); s.startDraft(step) }
}
}
window.addEventListener('keydown', onKey)
return () => window.removeEventListener('keydown', onKey)
}, [onClose])
// Keep the selected node in view when it changes, sideways on a phone.
useEffect(() => {
const node = boardRef.current?.querySelector('[data-selected="true"]')
node?.scrollIntoView?.({ block: 'nearest', inline: 'center' })
}, [selected?.id])
if (!members || !selected || !layout) {
return (
<div className="studio-lineage" data-testid="studio-lineage">
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={onClose}><Icon name="arrow-left" /> {t('studio.lineage.back')}</button>
<div className="studio-empty"><b>{t('studio.lineage.goneTitle')}</b><span>{t('studio.lineage.goneBody')}</span></div>
</div>
)
}
const root = members[0]
const steps = [...(NEXT_STEPS[selected.type] || []), ...(selected.type === 'images' && upscalers.length ? [{ to: 'images', edge: 'upscale', supported: true }] : [])]
const takeOk = canTake(selected)
const branchStep = suggestion || steps.find(s => s.supported) || null
const draftInstalled = draft ? installed.has(draft.to) : true
const draftInfo = draft ? TYPE_INFO[draft.to] : null
const draftTextLike = draftInfo?.input === 'text'
const canOpenDraft = !!draft && draftInstalled && (!draftTextLike || draft.text.trim().length > 0)
return (
<div className="studio-lineage" data-testid="studio-lineage" data-count={members.length}>
<header className="studio-lineage__head">
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={onClose} data-testid="studio-back">
<Icon name="arrow-left" /> {t('studio.lineage.back')}
</button>
<div>
<h1 className="studio-lineage__title" title={root.title || undefined}>{clip(root.title, 72) || label(root.type)}</h1>
<p className="studio-lineage__sub">
{members.length > 1 ? t('studio.lineage.sub', { count: members.length }) : t('studio.lineage.subOne')}
{' · '}{relativeTime(members[members.length - 1].createdAt)}
</p>
</div>
</header>
<div className="studio-board" ref={boardRef} data-testid="studio-board" tabIndex={0} role="group" aria-label={t('studio.lineage.board')}>
<div className="studio-board__canvas" style={cssVars({ '--w': `${layout.width}px`, '--h': `${layout.height}px` })}>
<svg className="studio-edges" width={layout.width} height={layout.height} aria-hidden="true">
{layout.edges.map(e => (
<path key={e.id} d={e.d} data-chain={e.chain || undefined} data-ghost={e.ghost || undefined} />
))}
</svg>
{layout.edges.filter(e => e.label).map(e => (
<span key={`l:${e.id}`} className="studio-edge-label" data-chain={e.chain || undefined} data-ghost={e.ghost || undefined}
style={cssVars({ '--x': `${e.lx}px`, '--y': `${e.ly}px` })}>
{t(`studio.edges.${e.label}`)}
</span>
))}
{layout.nodes.map(n => {
const pos = cssVars({ '--x': `${n.x}px`, '--y': `${n.y}px`, '--w': `${n.w}px`, '--h': `${n.h}px` })
if (n.kind === 'source') {
return (
<div key={n.id} className="studio-node studio-node--source" style={pos} data-testid="lineage-source">
{n.input === 'text'
? <span className="studio-node__quote">{n.title}</span>
: <span className="studio-node__file"><Icon name={TYPE_INFO[n.type].source === 'image' ? 'image' : 'waveform'} /></span>}
<span className="studio-node__label">
<b><Icon name={n.input === 'text' ? 'pencil' : 'file'} />{n.input === 'text' ? t('studio.lineage.prompt') : t('studio.lineage.file')}</b>
<small>{relativeTime(n.createdAt)}</small>
</span>
</div>
)
}
if (n.kind === 'ghost') {
const missing = n.ghost.edge !== 'upscale' && !installed.has(n.ghost.to)
return (
<button
key={n.id}
type="button"
className="studio-node studio-node--ghost"
style={pos}
data-testid="lineage-ghost"
data-missing={missing || undefined}
aria-pressed={n.ghost.draft}
onClick={() => startDraft(steps.find(s => s.edge === n.ghost.edge))}
>
<b><Icon name={TYPE_ICON[n.ghost.to]} />{t(`studio.steps.${n.ghost.edge}.title`)}</b>
<small>{missing ? t('studio.lineage.needsModel', { type: label(n.ghost.to) }) : (modalities.find(m => m.key === n.ghost.to)?.installed[0] || '')}</small>
<span className="dk-badge dk-badge--accent">{n.ghost.draft ? t('studio.lineage.draft') : t('studio.lineage.suggested')}</span>
</button>
)
}
const m = n.item
const isSel = m.id === selected.id
return (
<button
key={n.id}
type="button"
className="studio-node studio-node--result"
style={pos}
data-testid="lineage-node"
data-type={m.type}
data-id={m.id}
data-selected={isSel}
data-rel={layout.chain.has(n.id) ? '1' : '0'}
aria-pressed={isSel}
onClick={() => { setSelectedId(m.id); setDraft(null) }}
>
<WorkThumb item={m} compact />
{isSel && <span className="studio-picked" aria-hidden="true"><Icon name="check" /></span>}
<span className="studio-node__label">
<b><Icon name={TYPE_ICON[m.type]} /><span>{m.title || t(`studio.work.untitled.${m.type}`)}</span></b>
<small>{m.model || label(m.type)}</small>
</span>
</button>
)
})}
</div>
</div>
<section className="studio-dock" data-open={expanded} data-testid="studio-dock" aria-label={t('studio.lineage.dock')}>
<div className="studio-dock__row">
<span className="studio-dock__thumb">
{draft ? <Icon name={TYPE_ICON[draft.to]} /> : <WorkThumb item={selected} compact />}
</span>
<span className="studio-dock__who">
{draft ? (
<>
<b>{t('studio.lineage.newBranch', { step: t(`studio.steps.${draft.edge}.title`) })}</b>
<small>{t('studio.lineage.fromResult', { title: selected.title || label(selected.type) })}</small>
</>
) : (
<>
<b>{selected.title || t(`studio.work.untitled.${selected.type}`)}</b>
<small>{[selected.model, selected.elapsedMs ? t('studio.lineage.took', { seconds: Math.round(selected.elapsedMs / 1000) }) : '', relativeTime(selected.createdAt)].filter(Boolean).join(' · ')}</small>
</>
)}
</span>
<span className="studio-dock__actions">
{draft ? (
<>
<button type="button" className="dk-btn dk-btn--ghost dk-btn--sm" onClick={() => setDraft(null)} data-testid="studio-draft-cancel">{t('studio.lineage.cancel')}</button>
{draft.edge !== 'upscale' && <button type="button" className="dk-btn dk-btn--primary dk-btn--sm" disabled={!canOpenDraft} onClick={openDraft} data-testid="studio-draft-open">
<Icon name="sparkles" /> {t('studio.lineage.openIn', { type: label(draft.to) })}
</button>}
</>
) : (
<>
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" disabled={!takeOk.ok} onClick={take} data-testid="studio-take" title={takeOk.ok ? undefined : t('studio.lineage.noInput')}>
<Icon name="repeat" /> {t('studio.lineage.take')}
</button>
<button type="button" className="dk-btn dk-btn--primary dk-btn--sm" disabled={!branchStep} onClick={() => startDraft(branchStep)} data-testid="studio-branch" title={branchStep ? undefined : t('studio.lineage.noBranch', { type: label(selected.type) })}>
<Icon name="git-branch" /> {t('studio.lineage.branch')}
</button>
</>
)}
<button type="button" className="dk-btn dk-btn--ghost dk-btn--icon dk-btn--sm" aria-expanded={expanded} aria-label={t('studio.lineage.details')} onClick={() => setExpanded(v => !v)} data-testid="studio-dock-toggle">
<Icon name={expanded ? 'chevron-down' : 'chevron-up'} />
</button>
</span>
</div>
{(!takeOk.ok || !branchStep) && !draft && (
<p className="studio-dock__hint" data-testid="studio-dock-hint">
{!takeOk.ok && t('studio.lineage.noInput')}
{!takeOk.ok && !branchStep && ' '}
{!branchStep && t('studio.lineage.noBranch', { type: label(selected.type) })}
</p>
)}
{expanded && (draft?.edge === 'upscale' ? (
<UpscaleComposer key={selected.id} source={selected} upscalers={upscalers} onSuccess={entry => {
const id = onAddImage(entry)
setSelectedId(id)
setDraft(null)
}} />
) : draft ? (
<div className="studio-dock__body studio-draft" data-testid="studio-draft">
<span className="studio-eyebrow">{t('studio.lineage.makeFrom')}</span>
<div className="studio-draft__targets" role="group" aria-label={t('studio.lineage.targets')}>
{steps.map(step => {
const bad = !step.supported
return (
<button
key={step.edge}
type="button"
className="dk-chip studio-type"
aria-pressed={draft.edge === step.edge}
data-missing={(step.edge !== 'upscale' && !installed.has(step.to)) || undefined}
disabled={bad}
data-testid={`draft-target-${step.edge}`}
onClick={() => startDraft(step)}
>
<Icon name={TYPE_ICON[step.to]} />{t(`studio.steps.${step.edge}.title`)}
</button>
)
})}
</div>
{!draftInstalled && <InstallNote key={draft.to} type={draft.to} typeLabel={label(draft.to)} onChanged={onModelsChanged} />}
{draftTextLike ? (
<>
<span className="studio-eyebrow">{t('studio.lineage.describe')}</span>
<textarea className="studio-prompt studio-prompt--boxed" rows={3} value={draft.text} aria-label={t('studio.lineage.describe')} onChange={(e) => setDraft(d => ({ ...d, text: e.target.value }))} />
</>
) : (
<p className="studio-draft__plain">{t(`studio.steps.${draft.edge}.plain`)}</p>
)}
<p className="studio-draft__meta">{t('studio.lineage.keeps')}</p>
</div>
) : (
<div className="studio-dock__body studio-detail" data-testid="studio-detail">
<div className="studio-detail__preview"><Preview item={selected} t={t} /></div>
<div className="studio-detail__info">
<span className="studio-eyebrow">{t('studio.lineage.promptLabel')}</span>
<p className="studio-detail__prompt">{selected.title || t(`studio.work.untitled.${selected.type}`)}</p>
<p className="studio-detail__meta">{[selected.model, selected.params?.size, selected.params?.speakers ? t('studio.lineage.speakers', { count: selected.params.speakers }) : ''].filter(Boolean).join(' · ')}</p>
<span className="studio-eyebrow">{t('studio.lineage.continue')}</span>
<ul className="studio-steps">
{steps.length === 0 && <li className="studio-steps__none">{t('studio.lineage.noBranch', { type: label(selected.type) })}</li>}
{steps.map(step => {
const bad = !step.supported
const missing = !bad && step.edge !== 'upscale' && !installed.has(step.to)
return (
<li key={step.edge}>
<button type="button" className="studio-step" disabled={bad} onClick={() => startDraft(step)} data-testid={`step-${step.edge}`} data-state={bad ? 'unsupported' : missing ? 'missing' : 'ready'}>
<span className="studio-step__icon"><Icon name={TYPE_ICON[step.to]} /></span>
<span className="studio-step__text">
<b>{t(`studio.steps.${step.edge}.title`)}</b>
<small>
{bad
? t('studio.lineage.cannotStart', { page: label(step.to), what: t(`studio.kinds.${step.source}`) })
: t(`studio.steps.${step.edge}.why`)}
</small>
</span>
{!bad && <span className={`dk-badge ${missing ? 'dk-badge--warn' : 'dk-badge--ok'}`}>{missing ? t('studio.lineage.needsModel', { type: label(step.to) }) : t('studio.lineage.ready')}</span>}
</button>
</li>
)
})}
</ul>
</div>
<div className="studio-detail__foot">
<label className="studio-detail__fav"><StarButton on={selected.favourite} onToggle={() => onToggleFavourite(selected.id)} /><span>{t('studio.work.favourite')}</span></label>
{selected.url && <a className="dk-btn dk-btn--ghost dk-btn--sm" href={selected.url} download><Icon name="download" /> {t('studio.lineage.download')}</a>}
</div>
</div>
))}
</section>
</div>
)
}
// The large preview in the dock. Pictures and clips play from the server's own
// file; a mesh shows the picture it came from, because the viewer lives in its
// workspace.
// eslint-disable-next-line no-unused-vars
function Preview({ item, t }) {
const kind = TYPE_INFO[item.type].kind
if (kind === 'image' && item.url) return <img src={item.url} alt={item.title} />
if (kind === 'video' && item.url) return <video src={item.url} controls preload="metadata" aria-label={item.title || t('studio.tabs.video')} />
if (kind === 'audio' && item.url) {
return (
<div className="studio-detail__audio">
<WorkThumb item={item} />
<audio src={item.url} controls preload="none" aria-label={item.title || t('studio.tabs.sound')} />
</div>
)
}
return <WorkThumb item={item} />
}
addons/studio/ui/src/components/studio/StarButton.jsx
+28
View File
@@ -0,0 +1,28 @@
import { useTranslation } from 'react-i18next'
// The favourite star. The kit's icon set has no star, so it is drawn here; it
// is one path and takes its colour from the text colour like every other icon.
export function StarGlyph({ className = '' }) {
return (
<svg className={`studio-star__glyph ${className}`.trim()} viewBox="0 0 24 24" aria-hidden="true" focusable="false">
<path d="M12 3.6l2.6 5.5 6 .8-4.4 4.2 1.1 6-5.3-2.9-5.3 2.9 1.1-6L3.4 9.9l6-.8z" />
</svg>
)
}
export default function StarButton({ on, onToggle, className = '' }) {
const { t } = useTranslation('media')
return (
<button
type="button"
className={`studio-star ${className}`.trim()}
aria-pressed={on}
aria-label={t('studio.work.favourite')}
title={t('studio.work.favourite')}
data-testid="studio-star"
onClick={(e) => { e.stopPropagation(); e.preventDefault(); onToggle() }}
>
<StarGlyph />
</button>
)
}
addons/studio/ui/src/components/studio/StudioComposer.jsx
+258
View File
@@ -0,0 +1,258 @@
import { useEffect, useMemo, useRef } from 'react'
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
// eslint-disable-next-line no-unused-vars
import InstallNote from './InstallNote'
import { useResources } from '../../hooks/useResources'
import { useModelNeed } from '../../hooks/useModelSuggestion'
import { modelBudget } from '../../utils/modelBudget'
import { fitFor, gbLabel, gbNumber } from '../../utils/modelLedger'
import {
NEXT_STEPS, TYPE_ICON, TYPE_INFO, TYPE_ORDER, handoffPath, suggestType,
} from '../../utils/studioWork'
// The edge a hand-off carries when the chosen source feeds this type.
const EDGE_FOR = { images: 'variation', video: 'animate', threed: 'to-3d', transform: 'transform', diarization: 'diarize' }
// Which results can start which type.
function eligibleSources(type, items) {
const wanted = TYPE_INFO[type].source
if (!wanted) return []
const from = Object.keys(NEXT_STEPS).filter(k => NEXT_STEPS[k].some(s => s.supported && s.to === type && s.edge === EDGE_FOR[type]))
return items.filter(i => from.includes(i.type) && i.url).slice(0, 8)
}
function isTyping(el) {
if (!el) return false
const tag = el.tagName
return tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT' || el.isContentEditable
}
// "What do you want to make?"
//
// One box. The workspaces are the tabs above it, so the composer carries no
// type control of its own: it starts on the first type with a model, shows
// which type it will open, and suggests another from the words as a quiet hint
// (taken with the link or Alt+Enter, or Alt+1..7 to pick directly). Typing
// never switches by itself. Generate does not run anything here: it opens the
// workspace with the prompt and the options the workspace accepts.
export default function StudioComposer({
draft, setDraft, modalities, items, onHandoff, onModelsChanged, defaultModel,
}) {
const { t } = useTranslation('media')
const textRef = useRef(null)
const { resources } = useResources(30000)
const types = useMemo(() => TYPE_ORDER.filter(k => modalities.some(m => m.key === k)), [modalities])
const infoFor = (key) => modalities.find(m => m.key === key)
// Until one is picked, open on the first type that has a model, so a machine
// that can only read aloud does not open on a type it cannot run.
const firstReady = types.find(k => (infoFor(k)?.installed.length || 0) > 0)
const type = types.includes(draft.type) ? draft.type : (firstReady || types[0])
const modality = infoFor(type)
const info = TYPE_INFO[type]
const installed = modality?.installed || []
const missing = installed.length === 0
const model = installed.includes(draft.models[type]) ? draft.models[type] : (defaultModel(type, installed) || '')
const label = (key) => t(`studio.tabs.${key}`)
const guess = useMemo(() => {
const g = suggestType(draft.text)
return g && g !== type && types.includes(g) ? g : null
}, [draft.text, type, types])
const sources = useMemo(() => eligibleSources(type, items), [type, items])
const source = sources.find(s => s.id === draft.sourceId) || null
const need = useModelNeed(missing ? '' : model)
const fit = need ? fitFor(need, modelBudget(resources)) : null
const textLike = info.input === 'text'
const canGo = !missing && (!textLike || draft.text.trim().length > 0)
const why = missing ? t('studio.composer.whyModel', { type: label(type) })
: (textLike && !draft.text.trim()) ? t('studio.composer.whyPrompt') : ''
const set = (patch) => setDraft(d => ({ ...d, ...patch }))
const pickType = (key) => setDraft(d => ({ ...d, type: key, sourceId: '' }))
const go = () => {
if (!canGo) return
onHandoff(handoffPath(type, {
prompt: textLike ? draft.text : '',
model,
size: info.sizes ? (draft.sizes[type] || info.defaultSize) : '',
count: info.maxCount ? draft.count : 0,
from: source?.id || '',
edge: source ? EDGE_FOR[type] : '',
}))
}
// Keys: "/" to the prompt, Alt+1..7 to a type, Ctrl+Enter to generate, Alt+Enter
// to take the suggestion. Held in a ref so one listener sees the latest state.
const keyState = useRef({})
keyState.current = { go, guess, types, textLike }
useEffect(() => {
const onKey = (e) => {
const s = keyState.current
if (e.key === '/' && !e.ctrlKey && !e.metaKey && !e.altKey && !isTyping(e.target)) {
e.preventDefault()
textRef.current?.focus()
return
}
if (e.altKey && !e.ctrlKey && !e.metaKey && /^Digit[1-7]$/.test(e.code)) {
const key = s.types[Number(e.code.slice(5)) - 1]
if (key) { e.preventDefault(); pickType(key) }
return
}
if ((e.ctrlKey || e.metaKey) && e.key === 'Enter') { e.preventDefault(); s.go(); return }
if (e.altKey && e.key === 'Enter' && s.guess) { e.preventDefault(); pickType(s.guess) }
}
window.addEventListener('keydown', onKey)
return () => window.removeEventListener('keydown', onKey)
// pickType only calls setDraft, which is stable.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [])
// The box grows with what is typed, up to a limit.
useEffect(() => {
const el = textRef.current
if (!el) return
el.style.height = 'auto'
el.style.height = `${Math.min(220, el.scrollHeight)}px`
}, [draft.text, type])
const starters = (t(`studio.starters.${type}`, { returnObjects: true }))
const starterList = Array.isArray(starters) ? starters : []
const guessMissing = guess && (infoFor(guess)?.installed.length || 0) === 0
const noneInstalled = modalities.every(m => m.installed.length === 0)
return (
<section className="studio-composer" aria-labelledby="studio-composer-title" data-testid="studio-composer" data-type={type} data-types={types.join(' ')} data-missing={missing || undefined}>
<div className="studio-composer__head">
<h2 id="studio-composer-title">{t('studio.composer.title')}</h2>
<span>{t('studio.composer.lede')}</span>
<span className="studio-composer__now" data-testid="studio-current-type">{label(type)}</span>
</div>
<div className="studio-composer__box">
{textLike ? (
<textarea
ref={textRef}
className="studio-prompt"
rows={2}
value={draft.text}
spellCheck={false}
aria-label={t('studio.composer.prompt')}
placeholder={t(`studio.composer.placeholder.${type}`)}
onChange={(e) => set({ text: e.target.value })}
/>
) : (
<p className="studio-composer__file" data-testid="studio-file-lead">{t(`studio.composer.fileLead.${type}`)}</p>
)}
<div className="studio-hint" aria-live="polite" data-testid="studio-hint">
{guess && !guessMissing && (
<>
<span>{t('studio.composer.soundsLike', { type: label(guess) })}</span>
<button type="button" className="studio-link" data-testid="studio-switch" onClick={() => pickType(guess)}>{t('studio.composer.switchTo', { type: label(guess) })}</button>
<kbd className="dk-kbd">Alt Enter</kbd>
</>
)}
{guess && guessMissing && (
<>
<span>{t('studio.composer.soundsLikeMissing', { type: label(guess) })}</span>
<button type="button" className="studio-link" data-testid="studio-switch" onClick={() => pickType(guess)}>{t('studio.composer.seeNeeds')}</button>
</>
)}
</div>
</div>
{textLike && !draft.text && starterList.length > 0 && (
<div className="studio-starters" data-testid="studio-starters">
<span className="studio-eyebrow">{t('studio.composer.starters')}</span>
{starterList.map(s => (
<button key={s.label} type="button" className="dk-chip dk-chip--sm studio-starter" onClick={() => { set({ text: s.text }); textRef.current?.focus() }}>{s.label}</button>
))}
</div>
)}
{missing && <InstallNote key={type} type={type} typeLabel={label(type)} onChanged={onModelsChanged} className={noneInstalled ? 'studio-note--first' : ''} />}
{missing && noneInstalled && <p className="studio-firstrun" data-testid="studio-first-run">{t('studio.composer.firstRun')}</p>}
<div className="studio-options">
<span className="dk-select-wrap studio-opt">
<select
className="dk-select dk-input--mono"
aria-label={t('studio.composer.model')}
value={model}
disabled={missing}
data-testid="studio-model"
onChange={(e) => setDraft(d => ({ ...d, models: { ...d.models, [type]: e.target.value } }))}
>
{missing ? <option value="">{t('studio.composer.noModel')}</option> : installed.map(id => <option key={id} value={id}>{id}</option>)}
</select>
</span>
{info.sizes && (
<span className="dk-select-wrap studio-opt">
<select
className="dk-select dk-input--mono"
aria-label={t('studio.composer.size')}
value={draft.sizes[type] || info.defaultSize}
data-testid="studio-size"
onChange={(e) => setDraft(d => ({ ...d, sizes: { ...d.sizes, [type]: e.target.value } }))}
>
{info.sizes.map(s => <option key={s} value={s}>{s.replace('x', ' × ')}</option>)}
</select>
</span>
)}
{info.maxCount && (
<span className="dk-select-wrap studio-opt">
<select
className="dk-select"
aria-label={t('studio.composer.count')}
value={draft.count}
data-testid="studio-count"
onChange={(e) => set({ count: Number(e.target.value) })}
>
{Array.from({ length: info.maxCount }, (_, i) => i + 1).map(n => <option key={n} value={n}>{t('studio.composer.countOption', { count: n })}</option>)}
</select>
</span>
)}
{info.source && (
<span className="dk-select-wrap studio-opt studio-opt--source">
<select
className="dk-select"
aria-label={t('studio.composer.startFrom')}
value={draft.sourceId}
data-testid="studio-source"
onChange={(e) => set({ sourceId: e.target.value })}
>
<option value="">{info.needsSource ? t('studio.composer.sourceOnPage') : t('studio.composer.sourceNone')}</option>
{sources.map(s => <option key={s.id} value={s.id}>{(s.title || t(`studio.work.untitled.${s.type}`)).slice(0, 48)}</option>)}
</select>
</span>
)}
{fit && (
<span className="studio-fit" data-fit={fit.state} data-testid="studio-fit">
{fit.state === 'fits' && t('studio.composer.fit.fits', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
{fit.state === 'spill' && t('studio.composer.fit.spill', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
{fit.state === 'over' && t('studio.composer.fit.over', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
</span>
)}
<span className="studio-go">
{why && <span className="studio-go__why" data-testid="studio-why">{why}</span>}
<button type="button" className="dk-btn dk-btn--primary" disabled={!canGo} onClick={go} data-testid="studio-generate">
<Icon name="sparkles" />
{textLike ? t('studio.composer.generate') : t('studio.composer.openType', { type: label(type) })}
<kbd className="dk-kbd">Ctrl Enter</kbd>
</button>
</span>
</div>
</section>
)
}
addons/studio/ui/src/components/studio/Timeline.jsx
+31
View File
@@ -0,0 +1,31 @@
import { useTranslation } from 'react-i18next'
import { axisTicks, clock } from '../../utils/diarization'
import { cssVars } from '../../utils/modelLedger'
// Who spoke when: one lane per speaker, a bar for each stretch of speech, and a
// time axis. Drawn from the run's own segments; nothing here is estimated.
// The segment list under it is the same data as text.
//
// rows speakerRows(result)
// length the run's length in seconds
export default function Timeline({ rows, length }) {
const { t } = useTranslation('media')
const ticks = axisTicks(length)
return (
<div className="ws-timeline" role="img" aria-label={t('studio.workspace.diarization.timeline', { count: rows.length, length: clock(length) })} data-testid="ws-timeline">
<div className="ws-timeline__axis" aria-hidden="true">
{ticks.map(tick => <span key={tick.at} style={cssVars({ '--at': `${tick.pct}%` })}>{clock(tick.at)}</span>)}
</div>
{rows.map(row => (
<div key={row.label} className="ws-timeline__lane" data-speaker={row.index % 6} aria-hidden="true">
<span className="ws-timeline__who"><i />{row.name || row.id}</span>
<span className="ws-timeline__track">
{row.segments.map((seg, i) => (
<b key={seg.id ?? i} style={cssVars({ '--from': `${(Number(seg.start) / length) * 100}%`, '--len': `${Math.max(0.4, ((Number(seg.end) - Number(seg.start)) / length) * 100)}%` })} />
))}
</span>
</div>
))}
</div>
)
}
addons/studio/ui/src/components/studio/UpscaleComposer.jsx
+62
View File
@@ -0,0 +1,62 @@
// SPDX-License-Identifier: MIT
import { useEffect, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { imageApi } from '../../utils/api'
import { imageDimensions, sourceImageFile } from '../../utils/upscale'
export default function UpscaleComposer({ source, upscalers, onSuccess }) {
const { t } = useTranslation('media')
const [model, setModel] = useState(upscalers[0]?.id || '')
const [loading, setLoading] = useState(false)
const [error, setError] = useState('')
const active = useRef(true)
const running = useRef(false)
useEffect(() => { active.current = true; return () => { active.current = false } }, [])
const selected = upscalers.find(m => m.id === model) || upscalers[0]
const submit = async (event) => {
event.preventDefault()
if (!selected || running.current) return
running.current = true
setLoading(true)
setError('')
try {
const image = await sourceImageFile(source.url)
const sourceDimensions = await imageDimensions(image)
if (!active.current) return
const response = await imageApi.upscale({ model: selected.id, image, scale: selected.upscaleScale })
const results = (response?.data || []).map(r => ({ url: r.url || (r.b64_json ? `data:image/png;base64,${r.b64_json}` : '') }))
if (!results.length || results.some(r => !r.url)) throw new Error(t('studio.upscale.noResult'))
// A result that cannot be decoded is a failed run: never record guessed
// dimensions or an unusable child. The endpoint's dimensions are ignored.
const dimensions = await Promise.all(results.map(r => imageDimensions(r.url)))
if (!active.current) return
onSuccess({ model: selected.id, params: { scale: selected.upscaleScale, sourceDimensions, outputDimensions: dimensions[0] }, results, parentId: source.id, edge: 'upscale' })
} catch (err) {
if (active.current) setError(t('studio.upscale.failed', { message: err.message }))
} finally {
running.current = false
if (active.current) setLoading(false)
}
}
return (
<form className="studio-dock__body studio-detail" onSubmit={submit} data-testid="upscale-composer" aria-busy={loading}>
<div className="studio-detail__preview"><img src={source.url} alt={source.title || t('studio.upscale.source')} /></div>
<div className="studio-detail__info">
<label className="studio-eyebrow">{t('studio.upscale.model')}
<select className="dk-input" value={selected?.id || ''} onChange={e => setModel(e.target.value)} disabled={loading}>
{upscalers.map(m => <option key={m.id} value={m.id}>{m.id}</option>)}
</select>
</label>
<label className="studio-eyebrow">{t('studio.upscale.scale')}
<input className="dk-input" value={selected ? `${selected.upscaleScale}×` : ''} readOnly />
</label>
{error && <p role="alert">{error}</p>}
<button className="dk-btn dk-btn--primary" type="submit" disabled={loading || !selected}>
{t(loading ? 'studio.upscale.loading' : 'studio.steps.upscale.title')}
</button>
</div>
</form>
)
}
addons/studio/ui/src/components/studio/WorkMasonry.jsx
+192
View File
@@ -0,0 +1,192 @@
import { useEffect, useMemo, useRef, useState } from 'react'
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
// eslint-disable-next-line no-unused-vars
import ConfirmDialog from '../ConfirmDialog'
// eslint-disable-next-line no-unused-vars
import WorkThumb from './WorkThumb'
// eslint-disable-next-line no-unused-vars
import StarButton, { StarGlyph } from './StarButton'
import { relativeTime } from '../../utils/format'
import {
TYPE_ICON, TYPE_ORDER, columnCount, countByType, dealColumns, filterWork, groupWork,
} from '../../utils/studioWork'
// "Your work": every result this browser has a record of, newest first, as a
// masonry. Related results (one made from another) stack into a project tile.
//
// Filters count results, not tiles, so "Images 10" means ten pictures even when
// four of them sit in one stack. Grouping only applies under All: a type or
// Favourites filter lists the results themselves, which is what the count says.
export default function WorkMasonry({
items, ready, types, filter, onFilter, group, onGroup, onOpen, onToggleFavourite, onClear,
}) {
const { t } = useTranslation('media')
const [confirming, setConfirming] = useState(false)
const [columns, setColumns] = useState(3)
const host = useRef(null)
const counts = useMemo(() => countByType(items), [items])
const visible = useMemo(() => filterWork(items, filter), [items, filter])
const tiles = useMemo(() => {
if (filter === 'all' && group) return groupWork(visible)
return visible.map(item => ({ kind: 'single', id: item.id, item, latest: item.createdAt }))
}, [visible, filter, group])
// Column count follows the width of the page, not the window, so the sidebar
// being open or shut changes it too.
useEffect(() => {
const el = host.current
if (!el || typeof ResizeObserver === 'undefined') return undefined
const update = () => setColumns(columnCount(el.clientWidth))
update()
const ro = new ResizeObserver(update)
ro.observe(el)
return () => ro.disconnect()
}, [])
const cols = useMemo(() => dealColumns(tiles, columns), [tiles, columns])
const tabs = ['all', 'favourites', ...TYPE_ORDER.filter(k => types.includes(k))]
const label = (key) => key === 'all' ? t('studio.work.all')
: key === 'favourites' ? t('studio.work.favourites')
: t(`studio.tabs.${key}`)
return (
<section className="studio-work" aria-labelledby="studio-work-title" data-testid="studio-work">
<div className="studio-work__head">
<h2 id="studio-work-title">{t('studio.work.title')}</h2>
<span className="studio-work__count" data-testid="studio-work-count">{t('studio.work.results', { count: items.length })}</span>
<span className="studio-work__tools">
{filter === 'all' && (
<label className="studio-work__group">
<button
type="button"
role="switch"
className="dk-switch"
aria-checked={group}
aria-label={t('studio.work.group')}
onClick={() => onGroup(!group)}
data-testid="studio-group-switch"
/>
<span>{t('studio.work.group')}</span>
</label>
)}
{items.length > 0 && (
<button type="button" className="dk-btn dk-btn--ghost dk-btn--sm" onClick={() => setConfirming(true)} data-testid="studio-clear-history">
<Icon name="trash" /> {t('studio.work.clear')}
</button>
)}
</span>
</div>
<div className="studio-filters" role="tablist" aria-label={t('studio.work.filters')}>
{tabs.map(key => (
<button
key={key}
type="button"
role="tab"
className="studio-filter"
aria-selected={filter === key}
data-filter={key}
onClick={() => onFilter(key)}
>
{key === 'favourites' && <StarGlyph className="studio-filter__icon" />}
{label(key)}
<span className="studio-filter__n">{counts[key]}</span>
</button>
))}
</div>
<div ref={host} className="studio-masonry-host">
{!ready ? (
<div className="studio-masonry" aria-busy="true" data-testid="studio-work-loading">
{Array.from({ length: columns }, (_, c) => (
<div className="studio-col" key={c}>
<div className="dk-skeleton studio-skel studio-skel--tall" />
<div className="dk-skeleton studio-skel" />
</div>
))}
</div>
) : tiles.length === 0 ? (
<div className="studio-empty" data-testid="studio-work-empty">
<b>{items.length === 0 ? t('studio.work.emptyTitle') : t('studio.work.emptyFilterTitle', { filter: label(filter) })}</b>
<span>{items.length === 0 ? t('studio.work.emptyBody') : t('studio.work.emptyFilterBody')}</span>
</div>
) : (
<div className="studio-masonry" data-columns={columns}>
{cols.map((column, c) => (
<div className="studio-col" key={c}>
{column.map(tile => tile.kind === 'project'
? <ProjectTile key={tile.id} tile={tile} t={t} onOpen={onOpen} />
: <SingleTile key={tile.id} item={tile.item} t={t} onOpen={onOpen} onToggleFavourite={onToggleFavourite} />)}
</div>
))}
</div>
)}
</div>
<ConfirmDialog
open={confirming}
danger
title={t('studio.work.clearTitle')}
message={t('studio.work.clearBody')}
confirmLabel={t('studio.work.clearConfirm')}
onConfirm={() => { setConfirming(false); onClear() }}
onCancel={() => setConfirming(false)}
/>
</section>
)
}
// eslint-disable-next-line no-unused-vars
function SingleTile({ item, t, onOpen, onToggleFavourite }) {
return (
<div className="studio-tile" data-testid="work-tile" data-type={item.type} data-id={item.id}>
<button type="button" className="studio-tile__open" onClick={() => onOpen(item.id)} aria-label={t('studio.work.open', { title: item.title || t(`studio.tabs.${item.type}`) })}>
<span className="studio-tile__media">
<WorkThumb item={item} />
<span className="studio-badge"><Icon name={TYPE_ICON[item.type]} />{t(`studio.tabs.${item.type}`)}</span>
</span>
<span className="studio-tile__cap">
<span className="studio-tile__title">{item.title || t(`studio.work.untitled.${item.type}`)}</span>
<small>{[item.model, relativeTime(item.createdAt)].filter(Boolean).join(' · ')}</small>
</span>
</button>
<StarButton on={item.favourite} onToggle={() => onToggleFavourite(item.id)} className="studio-tile__star" />
</div>
)
}
// eslint-disable-next-line no-unused-vars
function ProjectTile({ tile, t, onOpen }) {
const chain = tile.items
const shown = chain.slice(0, 4)
const latest = chain[chain.length - 1]
return (
<div className="studio-stack" data-testid="work-project" data-id={tile.id} data-count={chain.length}>
<button type="button" className="studio-stack__open" onClick={() => onOpen(latest.id)} aria-label={t('studio.work.openProject', { title: tile.title || t('studio.work.project'), count: chain.length })}>
<span className="studio-stack__in">
<span className="studio-tile__media">
<WorkThumb item={tile.cover} />
<span className="studio-badge"><Icon name="layers" />{t('studio.work.nResults', { count: chain.length })}</span>
</span>
<span className="studio-tile__cap">
<span className="studio-tile__title studio-tile__title--project">{tile.title || t('studio.work.project')}</span>
<span className="studio-strip" aria-hidden="true">
{shown.map((m, i) => (
<span className="studio-strip__step" key={m.id}>
{i > 0 && <i className="studio-strip__dash" />}
<span className="studio-strip__glyph"><Icon name={TYPE_ICON[m.type]} /></span>
</span>
))}
{chain.length > shown.length && <span className="studio-strip__more">+{chain.length - shown.length}</span>}
<i className="studio-strip__dash studio-strip__dash--ghost" />
<span className="studio-strip__glyph studio-strip__glyph--ghost"><Icon name="plus" /></span>
</span>
<small>{relativeTime(tile.latest)}</small>
</span>
</span>
</button>
</div>
)
}
addons/studio/ui/src/components/studio/WorkThumb.jsx
+57
View File
@@ -0,0 +1,57 @@
import { useState } from 'react'
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
import { aspectOf, TYPE_ICON, TYPE_INFO, waveBars } from '../../utils/studioWork'
import { cssVars } from '../../utils/modelLedger'
// What a result looks like on a tile or a node.
//
// images the picture the server kept
// video its first frame, read from the file
// 3D the small picture it was made from, else a mesh placeholder
// audio a drawn waveform. It is a placeholder that is the same for the
// same result every time; it is not the sound
// diarization one bar per speaker the run found
//
// A picture whose file is gone from the server (the output folder was cleaned)
// falls back to the type's icon rather than a broken image.
export default function WorkThumb({ item, compact = false }) {
const { t } = useTranslation('media')
const [failed, setFailed] = useState(false)
const kind = TYPE_INFO[item.type].kind
const ratio = compact ? undefined : Math.min(1.8, Math.max(0.55, 1 / aspectOf(item)))
const frame = ratio ? cssVars({ '--studio-ratio': ratio }) : undefined
let body
if (kind === 'image' && item.url && !failed) {
body = <img src={item.url} alt="" loading="lazy" onError={() => setFailed(true)} />
} else if (kind === 'video' && item.url && !failed) {
// #t=0.1 asks for a frame past the black first one; preload=metadata keeps it to one small range request.
body = <video src={`${item.url}#t=0.1`} preload="metadata" muted playsInline tabIndex={-1} aria-hidden="true" onError={() => setFailed(true)} />
} else if (kind === 'mesh' && item.thumb) {
body = <img src={item.thumb} alt="" loading="lazy" />
} else if (kind === 'mesh') {
body = <Icon name="cube" className="studio-thumb__glyph" />
} else if (kind === 'audio') {
body = (
<span className="studio-wave" aria-hidden="true">
{waveBars(item.id, compact ? 28 : 44).map((h, i) => <i key={i} style={cssVars({ '--h': h })} />)}
</span>
)
} else if (kind === 'speakers') {
const n = Math.max(1, Math.min(6, Number(item.params?.speakers) || 1))
body = (
<span className="studio-lanes" aria-hidden="true">
{Array.from({ length: n }, (_, i) => <i key={i} data-speaker={i % 4} style={cssVars({ '--from': `${(i * 17) % 40}%`, '--to': `${60 + ((i * 23) % 40)}%` })} />)}
</span>
)
} else {
body = <Icon name={TYPE_ICON[item.type]} className="studio-thumb__glyph" title={failed ? t('studio.work.fileGone') : undefined} />
}
return (
<span className={`studio-thumb studio-thumb--${kind}${compact ? ' studio-thumb--compact' : ''}`} style={frame} data-failed={failed || undefined}>
{body}
</span>
)
}
addons/studio/ui/src/components/studio/Workspace.jsx
+564
View File
@@ -0,0 +1,564 @@
import { useEffect, useId, useMemo, useRef, useState } from 'react'
// eslint-disable-next-line no-unused-vars
import { Link } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import Icon from '../Icon'
// eslint-disable-next-line no-unused-vars
import ModelSelector from '../ModelSelector'
// eslint-disable-next-line no-unused-vars
import InstallNote from './InstallNote'
// eslint-disable-next-line no-unused-vars
import WorkThumb from './WorkThumb'
// eslint-disable-next-line no-unused-vars
import { StarGlyph } from './StarButton'
import { findParent } from '../../hooks/useWorkspace'
import { useResources } from '../../hooks/useResources'
import { useModelNeed } from '../../hooks/useModelSuggestion'
import { modelBudget } from '../../utils/modelBudget'
import { cssVars, fitFor, gbLabel, gbNumber } from '../../utils/modelLedger'
import { relativeTime } from '../../utils/format'
import { TYPE_ICON } from '../../utils/studioWork'
import './studio.css'
import './workspace.css'
// The shared frame of the seven Studio workspaces. A page keeps its own fields,
// endpoints and viewer; this file gives them one shape: a compose card (sources,
// prompt, a model chip, the essential options, an Advanced fold, an estimate and
// one action), a run area (a job card, a failure or the result with its
// toolbar), and a strip of recent results of the same type.
export function Workspace({ type, children }) {
return (
<div className="page-pad ws" data-testid="studio-workspace" data-type={type}>
{children}
</div>
)
}
// A labelled control. `changed` outlines it when "Re-run with edits" has moved
// it away from the take it started from.
export function Field({ label, htmlFor, changed, hint, children, className = '' }) {
return (
<div className={`ws-field ${className}`.trim()} data-changed={changed || undefined}>
{label && <label className="ws-label" htmlFor={htmlFor}>{label}</label>}
{children}
{hint && <p className="ws-hint">{hint}</p>}
</div>
)
}
// A native select drawn as a chip, for the few options that are always shown.
export function ChipSelect({ label, value, onChange, options, changed, mono = true, testId, disabled, id }) {
const items = options.map(o => (typeof o === 'object' ? o : { value: o, label: o }))
return (
<span className="dk-select-wrap ws-chip" data-changed={changed || undefined}>
<select
id={id}
className={`dk-select${mono ? ' dk-input--mono' : ''}`}
aria-label={label}
value={value}
disabled={disabled}
data-testid={testId}
onChange={(e) => onChange(e.target.value)}
>
{items.map(o => <option key={o.value} value={o.value}>{o.label}</option>)}
</select>
</span>
)
}
// An on/off option drawn as a chip. Pressed shows the kit's check mark.
export function ToggleChip({ label, on, onChange, changed, testId }) {
return (
<button type="button" className="dk-chip ws-toggle" aria-pressed={on} data-changed={changed || undefined} data-testid={testId} onClick={() => onChange(!on)}>
{label}
</button>
)
}
// A short typed value drawn as a chip with its name in front: a duration, a
// frame rate.
export function ChipInput({ label, value, onChange, placeholder, type = 'text', step, min, changed, testId, width, id: given }) {
const generated = useId()
const id = given || generated
return (
<label className="ws-chip ws-chip--input" htmlFor={id} data-changed={changed || undefined}>
<span>{label}</span>
<input
id={id}
className="ws-chip__input"
type={type}
step={step}
min={min}
value={value}
placeholder={placeholder}
data-testid={testId}
data-width={width}
onChange={(e) => onChange(e.target.value)}
/>
</label>
)
}
// The model picker, drawn as a chip. It is the page's own ModelSelector
// (search, last model remembered); the fit sits in the estimate line.
export function ModelChip({ value, onChange, capability, options, loading, disabled, changed }) {
return (
<span className="ws-chip ws-chip--model" data-changed={changed || undefined}>
<ModelSelector
value={value}
onChange={onChange}
capability={capability}
options={options}
loading={loading}
disabled={disabled}
triggerClassName="ws-model"
/>
</span>
)
}
// Short example prompts, shown while the prompt is empty. The same starters the
// Studio front page offers for the type.
export function Starters({ type, onPick }) {
const { t } = useTranslation('media')
const list = t(`studio.starters.${type}`, { returnObjects: true })
if (!Array.isArray(list) || list.length === 0) return null
return (
<div className="ws-starters" data-testid="ws-starters">
<span className="studio-eyebrow">{t('studio.composer.starters')}</span>
{list.map(s => <button key={s.label} type="button" className="dk-chip dk-chip--sm" onClick={() => onPick(s.text)}>{s.label}</button>)}
</div>
)
}
// A source that is optional: a dashed chip that opens a file picker. With a
// file chosen it shows the name and a way to remove it.
export function SourceChip({ label, accept, multiple = false, name, count = 0, preview, onFiles, onClear, testId }) {
const id = useId()
const filled = !!name || count > 0
return (
<span className={`ws-source${filled ? ' ws-source--filled' : ''}`} data-testid={testId}>
<label className="ws-source__pick" htmlFor={id}>
{preview ? <img src={preview} alt="" /> : <Icon name={filled ? 'check' : 'plus'} />}
<span className="ws-source__text">
<b>{label}</b>
{filled && <small>{name || String(count)}</small>}
</span>
</label>
<input
id={id}
className="ws-source__input"
type="file"
accept={accept}
multiple={multiple}
aria-label={label}
onChange={(e) => { if (e.target.files?.length) onFiles(multiple ? [...e.target.files] : e.target.files[0]); e.target.value = '' }}
/>
{filled && onClear && (
<button type="button" className="ws-source__x" aria-label={`${label}: remove`} onClick={onClear}><Icon name="close" /></button>
)}
</span>
)
}
// A source the run cannot start without: a recording. A drop area with a Choose
// button; with a file chosen it shows the name and size.
export function SourceDrop({ label, accept, file, onFile, hint, testId, inputId }) {
const { t } = useTranslation('media')
const generated = useId()
const id = inputId || generated
const [hover, setHover] = useState(false)
const kb = file ? file.size / 1024 : 0
const size = !file ? '' : kb >= 1024 ? `${(kb / 1024).toFixed(1)} MB` : `${Math.max(1, Math.round(kb))} KB`
return (
<div
className={`ws-drop${file ? ' ws-drop--filled' : ''}${hover ? ' ws-drop--hover' : ''}`}
data-testid={testId}
onDragOver={(e) => { e.preventDefault(); setHover(true) }}
onDragLeave={() => setHover(false)}
onDrop={(e) => { e.preventDefault(); setHover(false); const f = e.dataTransfer.files?.[0]; if (f) onFile(f) }}
>
<Icon name={file ? 'check' : 'upload'} />
<span className="ws-drop__text">
<b>{file ? file.name : label}</b>
<small>{file ? size : (hint || t('studio.workspace.dropHint'))}</small>
</span>
<label className="dk-btn dk-btn--secondary dk-btn--sm" htmlFor={id}>{file ? t('studio.workspace.replace') : t('studio.workspace.choose')}</label>
<input
id={id}
className="ws-source__input"
type="file"
accept={accept}
aria-label={label}
onChange={(e) => { const f = e.target.files?.[0]; if (f) onFile(f); e.target.value = '' }}
/>
{file && <button type="button" className="ws-source__x" aria-label={`${label}: remove`} onClick={() => onFile(null)}><Icon name="close" /></button>}
</div>
)
}
// A source chip that opens a panel with its own input (record, paste).
export function PanelChip({ label, open, onToggle, filled, controls }) {
return (
<button type="button" className={`ws-source ws-source--btn${filled ? ' ws-source--filled' : ''}`} aria-expanded={open} aria-controls={controls} onClick={onToggle}>
<span className="ws-source__pick">
<Icon name={filled ? 'check' : 'plus'} />
<span className="ws-source__text"><b>{label}</b></span>
</span>
</button>
)
}
// The Advanced fold: what is inside is named on the bar, so closing it does not
// hide that something is set.
export function Fold({ label, summary, open, onToggle, id, children }) {
const body = useId()
const bodyId = id || body
return (
<div className="ws-fold" data-open={open || undefined}>
<button type="button" className="ws-fold__bar" aria-expanded={open} aria-controls={bodyId} onClick={onToggle}>
<b>{label}</b>
{summary && <span className="ws-fold__sum">{summary}</span>}
<Icon name={open ? 'chevron-up' : 'chevron-down'} />
</button>
{open && <div id={bodyId} className="ws-fold__body">{children}</div>}
</div>
)
}
// The memory a model needs and whether it fits, when the gallery can say. No
// time is promised: nothing in the server can say how long a run will take.
// eslint-disable-next-line no-unused-vars
function Estimate({ model }) {
const { t } = useTranslation('media')
const { resources } = useResources(30000)
const need = useModelNeed(model)
const fit = need ? fitFor(need, modelBudget(resources)) : null
if (!fit) return <span className="ws-estimate" data-testid="ws-estimate" />
return (
<span className="ws-estimate" data-testid="ws-estimate" data-fit={fit.state}>
<Icon name="memory" />
{fit.state === 'fits' && t('studio.composer.fit.fits', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
{fit.state === 'spill' && t('studio.composer.fit.spill', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
{fit.state === 'over' && t('studio.composer.fit.over', { need: gbLabel(fit.need), amount: gbNumber(fit.amount) })}
</span>
)
}
// The compose card.
//
// icon, title, lede what this workspace is
// handoff the hand-off note, when the page was opened from a result
// sources optional source chips (SourceChip, PanelChip)
// panels what a PanelChip opens
// children the prompt (and anything that belongs with it)
// options the model chip and the essential options
// fold the Advanced fold
// model the chosen model, for the fit line
// noModel { type, label, onChanged } when no model is installed
// changes what "Re-run with edits" has changed, from useWorkspace().changes
// submit { label, busyLabel, busy, disabled, why, icon }
export function ComposeCard({
ws, icon, title, lede, onSubmit, handoff, sources, panels, children, options, fold, model, noModel, changes = [], submit,
}) {
const { t } = useTranslation('media')
return (
<form className="ws-compose" onSubmit={onSubmit} ref={ws?.composeRef} data-testid="ws-compose" data-rerun={ws?.rerunning || undefined} noValidate>
<header className="ws-compose__head">
<h1 className="page-title ws-title">{icon && <Icon name={icon} />}{title}</h1>
{lede && <span className="ws-lede">{lede}</span>}
</header>
{handoff}
{sources && <div className="ws-sources" role="group" aria-label={t('studio.workspace.sourcesLabel')}>{sources}</div>}
{panels}
{children}
{noModel && (
<InstallNote key={noModel.type} type={noModel.type} typeLabel={noModel.label} onChanged={noModel.onChanged} className="ws-note" />
)}
{options && <div className="ws-options">{options}</div>}
{fold}
<footer className="ws-foot">
<Estimate model={model} />
{changes.length > 0 && (
<span className="ws-changes" data-testid="ws-changes">
<span>{t('studio.workspace.changedFrom')}</span>
{changes.map(c => <code key={c.key}>{c.text}{c.long ? ` ${t('studio.workspace.edited')}` : ''}</code>)}
</span>
)}
<span className="ws-go">
{submit.why && <span className="ws-go__why" data-testid="ws-why">{submit.why}</span>}
<button type="submit" className="dk-btn dk-btn--primary ws-go__btn" disabled={submit.disabled || submit.busy} aria-busy={submit.busy || undefined}>
{!submit.busy && <Icon name={submit.icon || 'sparkles'} />}
{submit.busy ? submit.busyLabel : (ws?.rerunning ? t('studio.workspace.runAgain') : submit.label)}
</button>
</span>
</footer>
</form>
)
}
// --- The run area -------------------------------------------------------------
export function RunArea({ children }) {
return <section className="media-result ws-run" data-testid="ws-run">{children}</section>
}
export function EmptyRun({ icon, text }) {
return (
<div className="ws-empty" data-testid="ws-empty">
<Icon name={icon} />
<p>{text}</p>
</div>
)
}
function useElapsed() {
const [seconds, setSeconds] = useState(0)
useEffect(() => {
const id = setInterval(() => setSeconds(s => s + 1), 1000)
return () => clearInterval(id)
}, [])
return `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, '0')}`
}
// While a request is out. The server reports no phase and no percentage on
// these endpoints, so the bar is indeterminate and the only number shown is the
// time that has really passed.
//
// label "Making your picture"
// detail model, size and count as the request carries them
// tiles how many placeholder tiles to draw (pictures), 0 for none
export function JobCard({ label, detail, tiles = 0, ratio = 1 }) {
const elapsed = useElapsed()
const n = Math.min(Math.max(tiles, 0), 4)
return (
<div className="ws-job" role="status" aria-live="polite" data-testid="ws-job">
<div className="ws-job__text">
<p className="ws-job__title"><Icon name="spinner" spin /> <span>{label}</span> <span className="ws-job__time">{elapsed}</span></p>
<span className="dk-progress ws-job__bar" role="progressbar" aria-label={label} data-indeterminate><span className="dk-progress-bar" /></span>
{detail && <p className="ws-job__detail">{detail}</p>}
</div>
{n > 0 && (
<div className={`ws-job__tiles ws-job__tiles--n${n}`} aria-hidden="true">
{Array.from({ length: n }, (_, i) => <span key={i} className="dk-skeleton dk-skeleton--block ws-job__tile" style={cssVars({ '--ratio': ratio })} />)}
</div>
)}
</div>
)
}
// A run that did not finish: what the server said, that nothing was lost, and
// one action.
export function FailedCard({ message, onRetry, retryLabel }) {
const { t } = useTranslation('media')
return (
<div className="ws-failed" role="alert" data-testid="ws-failed">
<Icon name="alert-circle" />
<div className="ws-failed__body">
<h3>{t('studio.workspace.failed.title')}</h3>
<p className="ws-failed__message">{message}</p>
<p className="ws-failed__note">{t('studio.workspace.failed.note')}</p>
<div className="ws-failed__row">
{onRetry && <button type="button" className="dk-btn dk-btn--primary dk-btn--sm" onClick={onRetry}><Icon name="repeat" /> {retryLabel || t('studio.workspace.failed.retry')}</button>}
<a className="ws-link" href="/app/traces?tab=backend">{t('studio.workspace.failed.traces')}</a>
</div>
</div>
</div>
)
}
// --- The result ---------------------------------------------------------------
// eslint-disable-next-line no-unused-vars
function UseInMenu({ ws, item }) {
const { t } = useTranslation('media')
const [open, setOpen] = useState(false)
const root = useRef(null)
const steps = ws.steps(item)
useEffect(() => {
if (!open) return undefined
const onDown = (e) => { if (root.current && !root.current.contains(e.target)) setOpen(false) }
const onKey = (e) => { if (e.key === 'Escape') setOpen(false) }
document.addEventListener('mousedown', onDown)
document.addEventListener('keydown', onKey)
return () => { document.removeEventListener('mousedown', onDown); document.removeEventListener('keydown', onKey) }
}, [open])
const none = steps.length === 0
return (
<span className="dk-anchor" ref={root}>
<button
type="button"
className="dk-btn dk-btn--secondary dk-btn--sm"
aria-haspopup="menu"
aria-expanded={open}
disabled={!item || none}
data-testid="ws-use-in"
title={none ? t('studio.lineage.noBranch', { type: t(`studio.tabs.${ws.type}`) }) : undefined}
onClick={() => setOpen(v => !v)}
>
<Icon name="arrow-right" /> {t('studio.workspace.useIn')}
</button>
{open && (
<div className="dk-menu dk-menu--end ws-menu" role="menu" aria-label={t('studio.workspace.useIn')}>
{none && <p className="ws-menu__none">{t('studio.lineage.noBranch', { type: t(`studio.tabs.${ws.type}`) })}</p>}
{steps.map(step => (
<button
key={step.edge}
type="button"
role="menuitem"
className="dk-menu-item ws-menu__item"
disabled={step.blocked}
data-testid={`use-in-${step.edge}`}
data-state={step.blocked ? 'unsupported' : step.missingModel ? 'missing' : 'ready'}
onClick={() => { setOpen(false); ws.sendTo(item, step) }}
>
<Icon name={TYPE_ICON[step.to]} />
<span className="ws-menu__text">
<b>{step.title}</b>
<small>
{step.blocked
? t('studio.lineage.cannotStart', { page: t(`studio.tabs.${step.to}`), what: t(`studio.kinds.${step.source}`) })
: step.missingModel ? t('studio.lineage.needsModel', { type: t(`studio.tabs.${step.to}`) }) : t(`studio.steps.${step.edge}.why`)}
</small>
</span>
</button>
))}
</div>
)}
</span>
)
}
// The result and what can be done with it.
//
// item the stored result, when there is one (Favourite, Use in and
// Lineage need it); a result that was not stored has only Download
// title the heading; defaults to the prompt
// meta small facts under the viewer: model, size, seed
// download { href, name, testId }
// onRerun load this result's values into the form
// actions extra toolbar buttons that belong to the type (exports)
export function ResultCard({ ws, item, title, meta = [], download, onRerun, actions, children, footer, testId = 'ws-result' }) {
const { t } = useTranslation('media')
const parent = useMemo(() => findParent(item), [item])
const take = item ? ws.canTake(item) : { ok: false }
const heading = title || item?.title || t(`studio.work.untitled.${ws.type}`)
const edgeLabel = item?.edge ? t(`studio.edges.${item.edge}`, { defaultValue: item.edge }) : ''
return (
<article className="ws-result" data-testid={testId} data-id={item?.id}>
<header className="ws-result__head">
<div className="ws-result__who">
<h2 title={heading}>{heading}</h2>
<p>{[t(`studio.tabs.${ws.type}`), item ? relativeTime(item.createdAt) : ''].filter(Boolean).join(' · ')}</p>
</div>
<div className="ws-result__tools" role="toolbar" aria-label={t('studio.workspace.tools')}>
{item && (
<button type="button" className="ws-tool" aria-pressed={item.favourite} aria-label={t('studio.work.favourite')} title={t('studio.work.favourite')} data-testid="ws-favourite" onClick={() => ws.toggleFavourite(item.id)}>
<StarGlyph />
</button>
)}
{download && (
<a className="ws-tool" href={download.href} download={download.name} aria-label={t('studio.workspace.download')} title={t('studio.workspace.download')} data-testid={download.testId || 'ws-download'}>
<Icon name="download" />
</a>
)}
{actions}
{item && <UseInMenu ws={ws} item={item} />}
{onRerun && (
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" disabled={!item || !take.ok} title={take.ok ? undefined : t('studio.lineage.noInput')} data-testid="ws-rerun" onClick={() => onRerun(item)}>
<Icon name="repeat" /> {t('studio.workspace.rerun')}
</button>
)}
{item && (
<button type="button" className="ws-tool" aria-label={t('studio.workspace.lineage')} title={t('studio.workspace.lineage')} data-testid="ws-lineage" onClick={() => ws.openLineage(item)}>
<Icon name="git-branch" />
</button>
)}
</div>
</header>
<div className="ws-result__view">{children}</div>
<footer className="ws-result__meta">
{meta.filter(Boolean).map((m, i) => <span key={i}>{m}</span>)}
{item?.parentId && (
parent
? <span data-testid="ws-from">{t('studio.workspace.from')} <Link to={`/app/studio?work=${encodeURIComponent(parent.id)}`}>{parent.title || t(`studio.work.untitled.${parent.type}`)}</Link> ({edgeLabel})</span>
: <span data-testid="ws-from">{t('studio.workspace.fromGone', { edge: edgeLabel })}</span>
)}
{footer}
</footer>
</article>
)
}
// A card in the result's shape for something that is not a result yet: the
// audio you chose, waiting to be transformed.
export function ViewCard({ title, sub, children, testId = 'ws-view' }) {
return (
<article className="ws-result" data-testid={testId}>
<header className="ws-result__head">
<div className="ws-result__who">
<h2 title={title}>{title}</h2>
{sub && <p>{sub}</p>}
</div>
</header>
<div className="ws-result__view">{children}</div>
</article>
)
}
// --- The strip ----------------------------------------------------------------
const STRIP_MAX = 24
// Recent results of this type, from the shared history. Click one to show it
// above; click it again to go back to the latest.
export function ResultsStrip({ ws, selectedId, activeId, onSelect, onDelete, onClear }) {
const { t } = useTranslation('media')
const [filter, setFilter] = useState('all')
const favouriteCount = ws.items.filter(i => i.favourite).length
const shown = (filter === 'favourites' ? ws.items.filter(i => i.favourite) : ws.items).slice(0, STRIP_MAX)
return (
<section className="ws-strip media-history" data-testid="media-history" aria-label={t('studio.workspace.strip.title')}>
<header className="ws-strip__head">
<h2>{t('studio.workspace.strip.title')}</h2>
<span className="ws-strip__count">{t('studio.work.results', { count: ws.items.length })}</span>
<div className="ws-strip__tools">
<div className="studio-filters ws-strip__filters" role="tablist" aria-label={t('studio.work.filters')}>
<button type="button" role="tab" className="studio-filter" aria-selected={filter === 'all'} onClick={() => setFilter('all')}>{t('studio.work.all')}</button>
<button type="button" role="tab" className="studio-filter" aria-selected={filter === 'favourites'} onClick={() => setFilter('favourites')}>
{t('studio.work.favourites')} <span className="studio-filter__n">{favouriteCount}</span>
</button>
</div>
{ws.items.length > 0 && (
<button type="button" className="media-history-clear-btn ws-strip__clear" title={t('history.clearTitle')} aria-label={t('history.clearTitle')} onClick={onClear}>
<Icon name="trash" />
</button>
)}
<Link className="ws-link" to="/app/studio">{t('studio.workspace.strip.allWork')} <Icon name="arrow-right" /></Link>
</div>
</header>
{shown.length === 0 ? (
<p className="ws-strip__empty media-history-empty">{filter === 'favourites' ? t('studio.workspace.strip.noFavourites') : t('history.empty')}</p>
) : (
<ul className="ws-strip__list">
{shown.map(item => (
<li key={item.id} className="ws-tile" data-testid="media-history-item" data-id={item.id} data-active={item.id === activeId || undefined}>
<button type="button" className="ws-tile__open" aria-pressed={item.id === selectedId} aria-label={item.title || t(`studio.work.untitled.${ws.type}`)} onClick={() => onSelect(item.id)}>
<WorkThumb item={item} compact />
<span className="ws-tile__cap">
<span className="ws-tile__title">{item.title || t(`studio.work.untitled.${ws.type}`)}</span>
<small>{relativeTime(item.createdAt)}</small>
</span>
</button>
{item.favourite && <span className="ws-tile__star" aria-hidden="true"><StarGlyph /></span>}
<button type="button" className="ws-tile__del" aria-label={t('history.deleteEntry')} title={t('history.deleteEntry')} data-testid="media-history-delete" onClick={() => onDelete(item.id)}>
<Icon name="close" />
</button>
</li>
))}
</ul>
)}
</section>
)
}
addons/studio/ui/src/components/studio/studio.css
+640
View File
@@ -0,0 +1,640 @@
/* ==========================================================================
Studio front page: the composer, your work, and the lineage board.
Built on the kit's chip, segmented, switch, badge, select, button, skeleton
and meter classes. What is here is what the kit does not say: the masonry,
the stack, the board with its edges, and the dock. Selection is an outline,
a check or a surface step, never a rail. Surfaces are flat with a hairline;
the only shadows are the kit's two. No glow.
========================================================================== */
.studio-front { max-width: 1240px; }
.studio-key-hint {
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
color: var(--dk-muted);
font-size: var(--dk-text-13);
}
.studio-eyebrow {
font: var(--dk-weight-semibold) 11px / 1 var(--dk-font-body);
letter-spacing: .08em;
text-transform: uppercase;
color: var(--dk-muted);
}
.studio-link {
padding: 0;
border: 0;
background: none;
color: var(--dk-accent-text);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1.4 var(--dk-font-body);
text-decoration: underline;
text-underline-offset: 3px;
text-decoration-thickness: 1px;
cursor: pointer;
}
.studio-link:hover { text-decoration-thickness: 2px; }
/* --- Type tabs ------------------------------------------------------------- */
/* The seven types, as one row above every Studio page. The current one is
outlined and lifted onto a card; there is no underline and no rail. */
.studio-tabs {
display: flex;
flex-wrap: wrap;
gap: var(--dk-space-1);
padding: var(--dk-space-4) var(--spacing-xl) 0;
}
.studio-tab {
position: relative; /* contains the hidden state text so it cannot widen the scrolling row */
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
height: 38px;
padding: 0 var(--dk-space-3);
border: 0;
border-radius: var(--dk-radius-3);
background: transparent;
color: var(--dk-muted);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-body);
cursor: pointer;
transition: background-color var(--dk-dur-1) ease, color var(--dk-dur-1) ease, transform var(--dk-dur-1) var(--dk-ease-out);
}
.studio-tab:hover { background: var(--dk-hover); color: var(--dk-text); }
.studio-tab:active { transform: scale(var(--dk-press-scale)); }
.studio-tab:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.studio-tab .lai-icon { font-size: 16px; }
.studio-tab-active, .studio-tab-active:hover {
background: var(--dk-card);
color: var(--dk-text);
box-shadow: inset 0 0 0 1.5px var(--dk-text);
}
@media (max-width: 720px) {
.studio-tabs { flex-wrap: nowrap; overflow-x: auto; padding-left: var(--dk-space-4); padding-right: var(--dk-space-4); scrollbar-width: none; }
.studio-tab { flex: none; }
}
/* --- Composer ------------------------------------------------------------ */
.studio-composer {
background: var(--dk-card);
border-radius: var(--dk-radius-4);
box-shadow: inset 0 0 0 1px var(--dk-edge);
padding-bottom: var(--dk-space-4);
transition: box-shadow var(--dk-dur-2) var(--dk-ease-out);
}
.studio-composer:focus-within { box-shadow: inset 0 0 0 1px var(--dk-control-edge); }
.studio-composer__head {
display: flex;
align-items: baseline;
justify-content: space-between;
flex-wrap: wrap;
gap: var(--dk-space-3);
padding: var(--dk-space-4) var(--dk-space-5) 0;
}
.studio-composer__head h2 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-17) / 1.2 var(--dk-font-display); letter-spacing: -.01em; }
.studio-composer__head span { color: var(--dk-muted); font-size: var(--dk-text-13); }
.studio-composer__head .studio-composer__now { margin-left: auto; font: var(--dk-weight-semibold) var(--dk-text-12) / 1 var(--dk-font-mono); letter-spacing: .06em; text-transform: uppercase; }
.studio-type { height: 34px; }
.studio-composer__box { padding: var(--dk-space-2) var(--dk-space-5) 0; }
.studio-prompt {
display: block;
width: 100%;
min-height: 58px;
max-height: 220px;
padding: var(--dk-space-2) 0;
border: 0;
outline: 0;
resize: none;
background: transparent;
color: var(--dk-text);
font: 450 var(--dk-text-17) / 1.45 var(--dk-font-body);
}
.studio-prompt::placeholder { color: var(--dk-muted); }
.studio-prompt--boxed {
min-height: 84px;
padding: var(--dk-space-3);
border-radius: var(--dk-radius-2);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-control-edge);
font-size: var(--dk-text-15);
}
.studio-prompt--boxed:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.studio-prompt:focus-visible:not(.studio-prompt--boxed) { outline: none; }
.studio-composer__file {
margin: 0;
padding: var(--dk-space-4) 0 var(--dk-space-2);
max-width: 62ch;
color: var(--dk-muted);
font-size: var(--dk-text-15);
}
.studio-hint {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: var(--dk-space-2);
min-height: 20px;
color: var(--dk-muted);
font-size: var(--dk-text-13);
}
.studio-hint:empty { min-height: 0; }
.studio-hint b { color: var(--dk-text); font-weight: var(--dk-weight-semibold); }
.studio-starters {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: var(--dk-space-2);
padding: var(--dk-space-2) var(--dk-space-5) 0;
}
.studio-starters .studio-eyebrow { margin-right: var(--dk-space-1); }
.studio-options {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: var(--dk-space-2);
margin-top: var(--dk-space-4);
padding: var(--dk-space-4) var(--dk-space-4) 0;
border-top: 1px solid var(--dk-edge);
}
.studio-opt { max-width: 100%; }
.studio-opt .dk-select { max-width: 100%; text-overflow: ellipsis; font-size: var(--dk-text-13); }
.studio-opt--source .dk-select { max-width: 240px; }
.studio-fit { color: var(--dk-muted); font-size: var(--dk-text-12); }
.studio-fit[data-fit="spill"] { color: var(--dk-warn-text); }
.studio-fit[data-fit="over"] { color: var(--dk-error-text); }
.studio-go { display: inline-flex; align-items: center; gap: var(--dk-space-3); margin-left: auto; flex-wrap: wrap; justify-content: flex-end; }
.studio-go__why { color: var(--dk-muted); font-size: var(--dk-text-13); }
.studio-go .dk-kbd { margin-left: var(--dk-space-1); }
.studio-firstrun { margin: var(--dk-space-3) var(--dk-space-5) 0; max-width: 62ch; color: var(--dk-muted); font-size: var(--dk-text-13); }
/* --- Notes: install and error -------------------------------------------- */
.studio-note {
display: grid;
gap: var(--dk-space-2);
margin: var(--dk-space-3) var(--dk-space-4) 0;
padding: var(--dk-space-3) var(--dk-space-4);
border-radius: var(--dk-radius-3);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
}
.studio-note h3 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); }
.studio-note p { margin: 0; max-width: 66ch; color: var(--dk-text); font-size: var(--dk-text-13); line-height: 1.5; }
.studio-note__model { font-family: var(--dk-font-mono); font-size: var(--dk-text-12); overflow-wrap: anywhere; }
.studio-note__row { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); }
.studio-note__skeleton { width: 60%; }
.studio-note__progress { display: inline-flex; align-items: center; gap: var(--dk-space-2); font-size: var(--dk-text-13); }
.studio-note__meter { width: 160px; height: 8px; }
.studio-note__error { color: var(--dk-error-text); font-size: var(--dk-text-13); }
.studio-note--standalone { margin: 0; padding: var(--dk-space-4) var(--dk-space-5); border-radius: var(--dk-radius-4); }
.studio-note--error { background: var(--dk-error-wash); box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--dk-error) 40%, transparent); }
.studio-note--error h3 { color: var(--dk-error-text); }
/* --- Your work ------------------------------------------------------------ */
.studio-work { margin-top: var(--dk-space-6); }
.studio-work__head { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); }
.studio-work__head h2 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-21) / 1.2 var(--dk-font-display); letter-spacing: -.015em; }
.studio-work__count { color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
.studio-work__tools { display: inline-flex; align-items: center; gap: var(--dk-space-3); margin-left: auto; flex-wrap: wrap; }
.studio-work__group { display: inline-flex; align-items: center; gap: var(--dk-space-2); color: var(--dk-text); font-size: var(--dk-text-13); cursor: pointer; }
.studio-filters { display: flex; flex-wrap: wrap; gap: var(--dk-space-1); margin: var(--dk-space-3) 0 var(--dk-space-4); }
.studio-filter {
display: inline-flex;
align-items: center;
gap: 6px;
height: 30px;
padding: 0 11px;
border: 0;
border-radius: var(--dk-radius-1);
background: transparent;
color: var(--dk-text);
font: 500 var(--dk-text-13) / 1 var(--dk-font-body);
cursor: pointer;
transition: background-color var(--dk-dur-1) ease, color var(--dk-dur-1) ease, transform var(--dk-dur-1) var(--dk-ease-out);
}
.studio-filter:hover { background: var(--dk-hover); }
.studio-filter:active { transform: scale(var(--dk-press-scale)); }
.studio-filter[aria-selected="true"] { background: var(--dk-inverse); color: var(--dk-on-inverse); font-weight: var(--dk-weight-semibold); }
.studio-filter__n { font: 500 11px / 1 var(--dk-font-mono); opacity: .8; }
.studio-filter__icon { width: 14px; height: 14px; }
.studio-masonry { display: flex; align-items: flex-start; gap: var(--dk-space-4); }
.studio-col { flex: 1 1 0; min-width: 0; display: flex; flex-direction: column; gap: var(--dk-space-4); }
.studio-masonry-host { min-width: 0; }
.studio-skel { border-radius: var(--dk-radius-3); height: 160px; }
.studio-skel--tall { height: 240px; }
.studio-skel--composer { height: 220px; border-radius: var(--dk-radius-4); }
.studio-skel--board { height: 360px; border-radius: var(--dk-radius-4); }
.studio-empty {
display: grid;
gap: var(--dk-space-1);
max-width: 560px;
padding: var(--dk-space-5) var(--dk-space-5);
border: 1.5px dashed var(--dk-control-edge);
border-radius: var(--dk-radius-4);
color: var(--dk-muted);
font-size: var(--dk-text-13);
}
.studio-empty b { color: var(--dk-text); font-size: var(--dk-text-15); }
/* --- Tiles ---------------------------------------------------------------- */
.studio-tile, .studio-stack { position: relative; animation: studio-in var(--dk-dur-3) var(--dk-ease-out) both; }
@keyframes studio-in { from { opacity: 0; transform: translateY(8px); } }
.studio-tile__open, .studio-stack__open {
display: block;
width: 100%;
padding: 0;
border: 0;
background: none;
color: inherit;
text-align: left;
font: inherit;
cursor: pointer;
}
.studio-tile__open {
overflow: hidden;
border-radius: var(--dk-radius-3);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
transition: transform var(--dk-dur-2) var(--dk-ease-out), box-shadow var(--dk-dur-2) var(--dk-ease-out);
}
.studio-tile__open:hover { box-shadow: inset 0 0 0 1px var(--dk-control-edge), var(--dk-shadow-float); }
.studio-tile__open:active, .studio-stack__open:active .studio-stack__in { transform: scale(.985); }
.studio-tile__media { position: relative; display: block; background: var(--dk-inset); }
.studio-badge {
position: absolute;
left: 8px;
top: 8px;
display: inline-flex;
align-items: center;
gap: 5px;
height: 24px;
padding: 0 8px;
border-radius: var(--dk-radius-1);
/* Over a picture, so a fixed dark chip rather than a theme colour. */
background: rgb(10 16 15 / .72);
color: #f1f5f4;
font: 500 11px / 1 var(--dk-font-mono);
}
.studio-badge .lai-icon { width: 13px; height: 13px; }
.studio-tile__cap { display: grid; gap: 4px; padding: 10px 12px 12px; }
.studio-tile__title {
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow: hidden;
font-size: var(--dk-text-13);
line-height: 1.35;
color: var(--dk-text);
overflow-wrap: anywhere;
}
.studio-tile__title--project { font-weight: var(--dk-weight-semibold); font-size: var(--dk-text-15); }
.studio-tile__cap small { color: var(--dk-muted); font: 500 11.5px / 1.2 var(--dk-font-mono); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.studio-tile__star { position: absolute; right: 8px; top: 8px; }
.studio-star {
display: grid;
place-items: center;
width: 28px;
height: 28px;
padding: 0;
border: 0;
border-radius: var(--dk-radius-1);
background: rgb(10 16 15 / .72);
color: #f1f5f4;
cursor: pointer;
transition: transform var(--dk-dur-1) var(--dk-ease-out), background-color var(--dk-dur-1) ease;
}
.studio-star:hover { background: rgb(10 16 15 / .88); }
.studio-star:active { transform: scale(var(--dk-press-scale)); }
.studio-star svg, .studio-star__glyph { width: 15px; height: 15px; fill: none; stroke: currentColor; stroke-width: 1.8; stroke-linejoin: round; }
.studio-star[aria-pressed="true"] svg { fill: currentColor; }
.studio-detail__fav .studio-star { background: var(--dk-inset); color: var(--dk-text); box-shadow: inset 0 0 0 1px var(--dk-edge); }
.studio-filter__icon.studio-star__glyph { width: 14px; height: 14px; }
/* Project stack: two paper layers behind the cover. */
.studio-stack { padding-bottom: 10px; }
.studio-stack::before, .studio-stack::after {
content: "";
position: absolute;
height: calc(100% - 10px);
border-radius: var(--dk-radius-3);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
transition: transform var(--dk-dur-2) var(--dk-ease-out);
}
.studio-stack::before { bottom: 0; left: 22px; right: 22px; background: var(--dk-inset); }
.studio-stack::after { bottom: 5px; left: 11px; right: 11px; }
.studio-stack:hover::after { transform: translateY(4px); }
.studio-stack:hover::before { transform: translateY(8px); }
.studio-stack__in {
position: relative;
z-index: 1;
display: block;
overflow: hidden;
border-radius: var(--dk-radius-3);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
transition: transform var(--dk-dur-2) var(--dk-ease-out), box-shadow var(--dk-dur-2) var(--dk-ease-out);
}
.studio-stack__open:hover .studio-stack__in { box-shadow: inset 0 0 0 1px var(--dk-control-edge), var(--dk-shadow-float); }
.studio-strip { display: flex; align-items: center; flex-wrap: wrap; gap: 0; row-gap: 4px; }
.studio-strip__step { display: inline-flex; align-items: center; }
.studio-strip__glyph {
display: grid;
place-items: center;
width: 24px;
height: 24px;
border-radius: var(--dk-radius-1);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-control-edge);
color: var(--dk-text);
font-size: 13px;
}
.studio-strip__glyph--ghost { background: transparent; box-shadow: none; border: 1.5px dashed var(--dk-control-edge); color: var(--dk-muted); }
.studio-strip__dash { display: inline-block; width: 8px; height: 1.5px; background: var(--dk-control-edge); }
.studio-strip__dash--ghost { background: none; border-top: 1.5px dashed var(--dk-control-edge); height: 0; }
.studio-strip__more { margin-left: 6px; color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
/* --- Thumbnails ------------------------------------------------------------ */
.studio-thumb {
position: relative;
display: block;
width: 100%;
overflow: hidden;
background: var(--dk-inset);
color: var(--dk-muted);
}
.studio-thumb--image, .studio-thumb--video { aspect-ratio: 1 / var(--studio-ratio, 1); }
.studio-thumb--image img, .studio-thumb--video video, .studio-thumb--mesh img {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
object-fit: cover;
display: block;
}
.studio-thumb--mesh, .studio-thumb--audio, .studio-thumb--speakers { aspect-ratio: 16 / 9; }
.studio-thumb__glyph { position: absolute; left: 50%; top: 50%; width: 34px; height: 34px; margin: -17px 0 0 -17px; }
.studio-thumb--mesh img { object-fit: contain; padding: 14px; }
.studio-thumb--compact { width: 100%; height: 84px; aspect-ratio: auto; flex: none; }
.studio-wave { position: absolute; inset: 0; display: flex; align-items: center; gap: 2px; padding: 0 12px; }
.studio-wave i { flex: 1 1 0; min-width: 1px; height: calc(var(--h) * 70%); border-radius: 2px; background: var(--dk-accent); opacity: .8; }
.studio-lanes { position: absolute; inset: 0; display: flex; flex-direction: column; justify-content: center; gap: 6px; padding: 0 14px; }
.studio-lanes i { position: relative; display: block; height: 8px; border-radius: 4px; background: var(--dk-edge); }
.studio-lanes i::after {
content: "";
position: absolute;
top: 0;
bottom: 0;
left: var(--from);
right: calc(100% - var(--to));
border-radius: 4px;
background: var(--dk-accent);
opacity: .85;
}
.studio-lanes i[data-speaker="1"]::after { background: var(--dk-series-2); }
.studio-lanes i[data-speaker="2"]::after { background: var(--dk-series-3); }
.studio-lanes i[data-speaker="3"]::after { background: var(--dk-series-4); }
/* --- Lineage view ----------------------------------------------------------- */
.studio-lineage { display: grid; gap: var(--dk-space-4); }
.studio-lineage__head { display: flex; align-items: flex-start; gap: var(--dk-space-4); flex-wrap: wrap; }
.studio-lineage__head > div { min-width: 0; flex: 1 1 240px; }
.studio-lineage__title { margin: 0; display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; overflow: hidden; font: var(--dk-weight-bold) 26px / 1.15 var(--dk-font-display); letter-spacing: -.02em; overflow-wrap: anywhere; }
.studio-lineage__sub { margin: 4px 0 0; color: var(--dk-muted); font-size: var(--dk-text-13); }
.studio-board {
position: relative;
overflow: auto;
border-radius: var(--dk-radius-4);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
-webkit-overflow-scrolling: touch;
}
.studio-board:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.studio-board__canvas { position: relative; width: var(--w); height: var(--h); min-width: 100%; }
.studio-edges { position: absolute; left: 0; top: 0; pointer-events: none; overflow: visible; }
.studio-edges path { fill: none; stroke: var(--dk-control-edge); stroke-width: 1.5; transition: stroke var(--dk-dur-2) ease, stroke-width var(--dk-dur-2) ease; }
.studio-edges path[data-chain] { stroke: var(--dk-text); stroke-width: 2.25; }
.studio-edges path[data-ghost] { stroke-dasharray: 5 5; }
.studio-edges path[data-ghost][data-chain] { stroke-width: 2; }
.studio-edge-label {
position: absolute;
left: var(--x);
top: var(--y);
transform: translate(-50%, -50%);
height: 22px;
padding: 0 9px;
border-radius: var(--dk-radius-1);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
color: var(--dk-muted);
font: 500 11px / 22px var(--dk-font-body);
white-space: nowrap;
pointer-events: none;
}
.studio-edge-label[data-chain] { color: var(--dk-text); font-weight: var(--dk-weight-semibold); box-shadow: inset 0 0 0 1px var(--dk-text); }
.studio-edge-label[data-ghost] { box-shadow: none; }
.studio-node {
position: absolute;
left: var(--x);
top: var(--y);
width: var(--w);
height: var(--h);
display: flex;
flex-direction: column;
padding: 0;
overflow: hidden;
border: 0;
border-radius: var(--dk-radius-3);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
color: var(--dk-text);
font: inherit;
text-align: left;
animation: studio-node-in var(--dk-dur-3) var(--dk-ease-out) both;
transition: transform var(--dk-dur-2) var(--dk-ease-spring), box-shadow var(--dk-dur-2) var(--dk-ease-out);
}
@keyframes studio-node-in { from { opacity: 0; transform: translateY(10px) scale(.96); } }
button.studio-node { cursor: pointer; }
button.studio-node:hover { box-shadow: inset 0 0 0 1px var(--dk-control-edge); }
button.studio-node:active { transform: scale(.98); }
.studio-node[data-selected="true"] { transform: translateY(-3px); box-shadow: inset 0 0 0 2px var(--dk-text), var(--dk-shadow-float); z-index: 3; }
.studio-node[data-rel="0"] .studio-thumb { opacity: .55; }
.studio-node__label { display: grid; gap: 3px; min-width: 0; padding: 7px 10px 8px; }
.studio-node__label b { display: flex; align-items: center; gap: 6px; min-width: 0; font: var(--dk-weight-semibold) 12px / 1.2 var(--dk-font-body); white-space: nowrap; overflow: hidden; }
.studio-node__label b span { overflow: hidden; text-overflow: ellipsis; }
.studio-node__label b .lai-icon { width: 14px; height: 14px; flex: none; color: var(--dk-muted); }
.studio-node__label small { color: var(--dk-muted); font: 500 10.5px / 1.2 var(--dk-font-mono); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.studio-node__quote {
flex: 1;
min-height: 0;
padding: 10px 12px 0;
display: -webkit-box;
-webkit-line-clamp: 4;
-webkit-box-orient: vertical;
max-height: calc(4 * 1.4em + 10px);
overflow: hidden;
color: var(--dk-text);
font: 450 11.5px / 1.4 var(--dk-font-body);
overflow-wrap: anywhere;
}
.studio-node__file { flex: 1; display: grid; place-items: center; color: var(--dk-muted); font-size: 30px; }
.studio-picked { position: absolute; right: 6px; top: 6px; display: grid; place-items: center; width: 22px; height: 22px; border-radius: var(--dk-radius-1); background: var(--dk-inverse); color: var(--dk-on-inverse); z-index: 2; }
.studio-picked .lai-icon { width: 13px; height: 13px; }
.studio-node--ghost {
justify-content: center;
gap: 6px;
padding: 12px;
overflow: visible;
background: transparent;
box-shadow: none;
border: 1.5px dashed var(--dk-control-edge);
}
button.studio-node--ghost:hover { background: var(--dk-card); box-shadow: none; }
.studio-node--ghost[aria-pressed="true"] { border-style: solid; border-color: var(--dk-text); background: var(--dk-card); box-shadow: var(--dk-shadow-float); }
.studio-node--ghost b { display: flex; align-items: center; gap: 7px; font: var(--dk-weight-semibold) 13px / 1.2 var(--dk-font-body); }
.studio-node--ghost small { color: var(--dk-muted); font: 500 11px / 1.35 var(--dk-font-body); overflow-wrap: anywhere; }
.studio-node--ghost .dk-badge { align-self: flex-start; }
/* --- Dock ------------------------------------------------------------------- */
.studio-dock {
position: sticky;
bottom: var(--dk-space-3);
z-index: var(--dk-z-sticky);
border-radius: var(--dk-radius-4);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-control-edge), var(--dk-shadow-float);
overflow: hidden;
}
.studio-dock__row { display: flex; align-items: center; gap: var(--dk-space-3); padding: 8px 10px; }
.studio-dock__thumb { flex: none; width: 76px; height: 48px; overflow: hidden; border-radius: var(--dk-radius-1); background: var(--dk-inset); display: grid; place-items: center; color: var(--dk-muted); font-size: 20px; }
.studio-dock__thumb .studio-thumb { height: 48px; }
.studio-dock__who { display: grid; gap: 2px; min-width: 0; flex: 1 1 auto; }
.studio-dock__who b { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font: var(--dk-weight-semibold) var(--dk-text-13) / 1.3 var(--dk-font-body); }
.studio-dock__who small { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--dk-muted); font: 500 11.5px / 1.2 var(--dk-font-mono); }
.studio-dock__actions { display: inline-flex; align-items: center; gap: var(--dk-space-2); flex: none; }
.studio-dock__hint { margin: 0; padding: 0 var(--dk-space-4) var(--dk-space-3); color: var(--dk-muted); font-size: var(--dk-text-12); }
.studio-dock__body { border-top: 1px solid var(--dk-edge); padding: var(--dk-space-4) var(--dk-space-4); max-height: min(60vh, 560px); overflow: auto; }
.studio-detail { display: grid; grid-template-columns: minmax(0, 340px) minmax(0, 1fr); gap: var(--dk-space-5); }
.studio-detail__preview { min-width: 0; border-radius: var(--dk-radius-3); overflow: hidden; background: var(--dk-inset); align-self: start; }
.studio-detail__preview img, .studio-detail__preview video { display: block; width: 100%; max-height: 360px; object-fit: contain; }
.studio-detail__audio { display: grid; gap: var(--dk-space-2); padding-bottom: var(--dk-space-2); }
.studio-detail__audio audio { width: calc(100% - 16px); margin: 0 8px; }
.studio-detail__info { display: grid; gap: var(--dk-space-2); align-content: start; min-width: 0; }
.studio-detail__prompt { margin: 0; font-size: var(--dk-text-15); overflow-wrap: anywhere; }
.studio-detail__meta { margin: 0 0 var(--dk-space-3); color: var(--dk-muted); font: 500 12px / 1.4 var(--dk-font-mono); }
.studio-detail__foot { grid-column: 1 / -1; display: flex; align-items: center; gap: var(--dk-space-4); }
.studio-detail__fav { display: inline-flex; align-items: center; gap: var(--dk-space-2); font-size: var(--dk-text-13); cursor: pointer; }
.studio-steps { display: grid; gap: var(--dk-space-1); margin: 0; padding: 0; list-style: none; }
.studio-steps__none { color: var(--dk-muted); font-size: var(--dk-text-13); }
.studio-step {
display: grid;
grid-template-columns: 36px minmax(0, 1fr) auto;
gap: var(--dk-space-3);
align-items: center;
width: 100%;
padding: 8px 8px;
border: 0;
border-radius: var(--dk-radius-2);
background: transparent;
color: var(--dk-text);
text-align: left;
font: inherit;
cursor: pointer;
transition: background-color var(--dk-dur-1) ease, transform var(--dk-dur-1) var(--dk-ease-out);
}
.studio-step:hover:not(:disabled) { background: var(--dk-hover); }
.studio-step:active:not(:disabled) { transform: scale(var(--dk-press-scale-nav)); }
.studio-step:disabled { cursor: not-allowed; opacity: .7; }
.studio-step__icon { display: grid; place-items: center; width: 36px; height: 36px; border-radius: var(--dk-radius-2); background: var(--dk-inset); color: var(--dk-text); }
.studio-step__text { display: grid; gap: 2px; min-width: 0; }
.studio-step__text b { font: var(--dk-weight-semibold) var(--dk-text-13) / 1.3 var(--dk-font-body); }
.studio-step__text small { color: var(--dk-muted); font-size: var(--dk-text-12); }
.studio-draft { display: grid; gap: var(--dk-space-3); }
.studio-draft__targets { display: flex; flex-wrap: wrap; gap: var(--dk-space-2); }
.studio-draft .studio-note { margin: 0; }
.studio-draft__plain { margin: 0; color: var(--dk-muted); font-size: var(--dk-text-13); max-width: 62ch; }
.studio-draft__meta { margin: 0; color: var(--dk-muted); font: 500 12px / 1.4 var(--dk-font-mono); }
/* --- Hand-off note on a workspace -------------------------------------------- */
.studio-handoff {
display: flex;
align-items: center;
gap: var(--dk-space-2);
margin-bottom: var(--dk-space-3);
padding: var(--dk-space-2) var(--dk-space-3);
border-radius: var(--dk-radius-2);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
color: var(--dk-text);
font-size: var(--dk-text-13);
}
.studio-handoff__text { flex: 1 1 auto; min-width: 0; overflow-wrap: anywhere; }
.studio-handoff--failed { background: var(--dk-warn-wash); box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--dk-warn) 40%, transparent); color: var(--dk-warn-text); }
/* --- Phone ---------------------------------------------------------------- */
@media (max-width: 720px) {
.studio-composer__head { padding: var(--dk-space-3) var(--dk-space-4) 0; }
.studio-composer__box, .studio-starters { padding-left: var(--dk-space-4); padding-right: var(--dk-space-4); }
.studio-options { padding-left: var(--dk-space-3); padding-right: var(--dk-space-3); }
.studio-go { margin-left: 0; width: 100%; justify-content: space-between; }
.studio-go .dk-btn { flex: 1 1 auto; }
.studio-go .dk-kbd { display: none; }
.studio-key-hint { display: none; }
.studio-opt--source .dk-select { max-width: 100%; }
.studio-work__tools { margin-left: 0; }
.studio-filters { flex-wrap: nowrap; overflow-x: auto; scrollbar-width: none; padding-bottom: 2px; }
.studio-filter { flex: none; }
.studio-lineage__title { font-size: 22px; }
.studio-dock__row { flex-wrap: wrap; }
.studio-dock__who { flex-basis: calc(100% - 100px); }
.studio-dock__actions { width: 100%; flex-wrap: wrap; justify-content: flex-end; }
.studio-dock__actions .dk-btn--sm:not(.dk-btn--icon) { flex: 1 1 auto; min-width: 0; padding: 0 10px; }
.studio-detail { grid-template-columns: minmax(0, 1fr); }
.studio-dock__thumb { width: 64px; }
}
@media (prefers-reduced-motion: reduce) {
.studio-tile, .studio-stack, .studio-node { animation: none; }
.studio-stack::before, .studio-stack::after, .studio-tile__open, .studio-stack__in, .studio-node, .studio-edges path { transition: none; }
}
addons/studio/ui/src/components/studio/workspace.css
+619
View File
@@ -0,0 +1,619 @@
/* ==========================================================================
Studio workspaces: one template for the seven types.
A compose card, a run area, a strip of recent results. Built on the kit's
button, select, chip, progress and skeleton classes; what is here is what the
kit does not say. Selection is an outline or a check, never a rail. Surfaces
are flat with a hairline and there is no glow.
========================================================================== */
.ws {
display: grid;
grid-template-columns: minmax(0, 1fr);
gap: var(--dk-space-5);
align-content: start;
max-width: 980px;
margin: 0;
}
.ws-link {
color: var(--dk-accent-text);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1.4 var(--dk-font-body);
text-decoration: underline;
text-underline-offset: 3px;
text-decoration-thickness: 1px;
display: inline-flex;
align-items: center;
gap: 4px;
}
.ws-link:hover { text-decoration-thickness: 2px; }
/* --- Compose card --------------------------------------------------------- */
.ws-compose {
display: grid;
grid-template-columns: minmax(0, 1fr);
gap: var(--dk-space-3);
padding: var(--dk-space-4) var(--dk-space-5) var(--dk-space-4);
border-radius: var(--dk-radius-4);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
transition: box-shadow var(--dk-dur-2) var(--dk-ease-out);
}
.ws-compose:focus-within { box-shadow: inset 0 0 0 1px var(--dk-control-edge); }
.ws-compose[data-rerun] { box-shadow: inset 0 0 0 1.5px var(--dk-accent); }
.ws-compose__head { display: flex; align-items: baseline; flex-wrap: wrap; gap: var(--dk-space-3); }
.ws-title {
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
margin: 0;
font: var(--dk-weight-semibold) var(--dk-text-17) / 1.2 var(--dk-font-display);
letter-spacing: -.01em;
}
.ws-title .lai-icon { color: var(--dk-muted); }
.ws-lede { color: var(--dk-muted); font-size: var(--dk-text-13); }
.ws-sources { display: flex; flex-wrap: wrap; gap: var(--dk-space-2); }
.ws-panel {
padding: var(--dk-space-3);
border-radius: var(--dk-radius-3);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
}
.ws-source {
position: relative;
display: inline-flex;
align-items: center;
min-height: 40px;
border-radius: var(--dk-radius-3);
border: 1.5px dashed var(--dk-control-edge);
background: transparent;
color: var(--dk-text);
font: inherit;
transition: background-color var(--dk-dur-1) ease;
}
.ws-source:hover, .ws-source:focus-within { background: var(--dk-hover); }
.ws-source--btn { cursor: pointer; padding: 0; }
.ws-source__pick {
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
min-height: 38px;
padding: 0 var(--dk-space-3);
cursor: pointer;
font-size: var(--dk-text-13);
}
.ws-source__pick img { width: 28px; height: 28px; border-radius: var(--dk-radius-1); object-fit: cover; }
.ws-source__text { display: grid; line-height: 1.2; text-align: left; }
.ws-source__text b { font-weight: var(--dk-weight-semibold); }
.ws-source__text small { max-width: 220px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--dk-muted); font: 500 11px / 1.3 var(--dk-font-mono); }
.ws-source--filled { border-style: solid; border-color: var(--dk-edge); background: var(--dk-inset); }
.ws-source--filled .ws-source__pick { padding-right: var(--dk-space-1); }
.ws-source__input { position: absolute; inline-size: 1px; block-size: 1px; opacity: 0; pointer-events: none; }
.ws-source:focus-within { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-source__x {
display: grid;
place-items: center;
width: 28px;
height: 28px;
margin-right: 6px;
padding: 0;
border: 0;
border-radius: 50%;
background: transparent;
color: var(--dk-muted);
cursor: pointer;
}
.ws-source__x:hover { background: var(--dk-hover); color: var(--dk-text); }
.ws-prompt,
.ws-compose .ws-prompt.textarea {
display: block;
width: 100%;
min-height: 96px;
padding: var(--dk-space-3);
border: 0;
border-radius: var(--dk-radius-3);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
color: var(--dk-text);
font: 450 var(--dk-text-15) / 1.5 var(--dk-font-body);
resize: vertical;
outline: 0;
}
.ws-prompt::placeholder { color: var(--dk-muted); opacity: 1; }
.ws-compose .ws-prompt[data-changed] { box-shadow: 0 0 0 2px var(--dk-accent); }
.ws-prompt:hover { box-shadow: inset 0 0 0 1px var(--dk-control-edge); }
.ws-prompt:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-starters { display: flex; flex-wrap: wrap; align-items: center; gap: var(--dk-space-2); }
.ws-note { margin: 0; }
.ws-options { display: flex; flex-wrap: wrap; align-items: center; gap: var(--dk-space-2); }
.ws-chip { max-width: 100%; }
.ws-chip .dk-select { height: 36px; max-width: 100%; text-overflow: ellipsis; font-size: var(--dk-text-13); font-weight: var(--dk-weight-semibold); }
.ws-chip[data-changed] .dk-select,
.ws-chip[data-changed] .ws-model { box-shadow: 0 0 0 2px var(--dk-accent); border-color: transparent; }
.ws-chip--input {
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
height: 36px;
padding: 0 var(--dk-space-3);
border: 1px solid var(--dk-control-edge);
border-radius: var(--dk-radius-2);
background: var(--dk-card);
color: var(--dk-muted);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-body);
cursor: text;
}
.ws-chip--input:hover { border-color: color-mix(in srgb, var(--dk-control-edge) 70%, var(--dk-text)); }
.ws-chip--input:focus-within { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-chip--input[data-changed] { box-shadow: 0 0 0 2px var(--dk-accent); border-color: transparent; }
.ws-chip__input {
width: 64px;
min-width: 0;
border: 0;
outline: 0;
background: transparent;
color: var(--dk-text);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-mono);
text-align: right;
}
.ws-chip__input::placeholder { color: var(--dk-muted); opacity: 1; }
/* A wide field holds a sentence-length placeholder ("Optional speaker or voice
ID"), which 120px cut off mid-word. Left-aligned text, and it gives way on a
phone instead of overflowing the row. */
.ws-chip__input[data-width="wide"] { width: 15rem; text-align: left; text-overflow: ellipsis; }
.ws-toggle { height: 36px; }
.ws-toggle[data-changed] { box-shadow: 0 0 0 2px var(--dk-accent); }
.ws-chip--model { position: relative; min-width: 0; }
.ws-chip--model > div { min-width: 0 !important; }
.ws-model {
display: inline-flex;
align-items: center;
gap: var(--dk-space-2);
height: 36px;
max-width: 100%;
padding: 0 var(--dk-space-3);
border: 1px solid var(--dk-control-edge);
border-radius: var(--dk-radius-2);
background: var(--dk-card);
color: var(--dk-text);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-mono);
cursor: pointer;
}
.ws-model:hover:not(:disabled) { background: var(--dk-hover); }
.ws-model:disabled { opacity: .5; cursor: not-allowed; }
.ws-model:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-model .searchable-select__label { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; text-align: left; }
.ws-model .searchable-select__chevron { flex: none; width: 14px; height: 14px; color: var(--dk-muted); }
/* Fields (Advanced fold, and fields of the types that have them inline). */
.ws-field { display: grid; gap: 6px; align-content: start; min-width: 0; border-radius: var(--dk-radius-2); }
/* The Home picker sizes its menu from the viewport, which assumes a 24px page
gutter. Inside the composer card the field sits further in, so the menu would
run past the right edge of a narrow screen. Size it from the field instead. */
.ws-field .home-menu { width: auto; min-width: min(400px, 100%); max-width: min(560px, 100%); }
.ws-field[data-changed] .dk-input,
.ws-field[data-changed] .dk-select,
.ws-field[data-changed] .ws-prompt,
.ws-field[data-changed] .textarea { box-shadow: 0 0 0 2px var(--dk-accent); border-color: transparent; }
.ws-label { font: var(--dk-weight-semibold) 11px / 1.2 var(--dk-font-body); letter-spacing: .07em; text-transform: uppercase; color: var(--dk-muted); }
.ws-hint { margin: 0; color: var(--dk-muted); font-size: var(--dk-text-12); line-height: 1.4; }
.ws-field .dk-input, .ws-field .dk-select { height: 38px; font-size: var(--dk-text-13); }
.ws-field .dk-textarea, .ws-field .textarea {
min-height: 64px;
padding: var(--dk-space-2) var(--dk-space-3);
border: 1px solid var(--dk-control-edge);
border-radius: var(--dk-radius-2);
background: var(--dk-card);
color: var(--dk-text);
font: var(--dk-weight-regular) var(--dk-text-13) / 1.5 var(--dk-font-body);
resize: vertical;
}
.ws-field .textarea:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: var(--dk-space-3); }
.ws-grid--wide { grid-template-columns: 1fr; }
.ws-check { display: inline-flex; align-items: center; gap: var(--dk-space-2); min-height: 38px; font-size: var(--dk-text-13); cursor: pointer; }
.ws-check input { width: 16px; height: 16px; accent-color: var(--dk-accent); }
/* A required source: a drop area. */
.ws-drop {
position: relative;
display: flex;
align-items: center;
gap: var(--dk-space-3);
padding: var(--dk-space-3) var(--dk-space-4);
border: 1.5px dashed var(--dk-control-edge);
border-radius: var(--dk-radius-3);
color: var(--dk-text);
transition: background-color var(--dk-dur-1) ease;
}
.ws-drop:focus-within { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
.ws-drop--hover { background: var(--dk-accent-wash); border-color: var(--dk-accent); }
.ws-drop--filled { border-style: solid; border-color: var(--dk-edge); background: var(--dk-inset); }
.ws-drop > .lai-icon { flex: none; font-size: 20px; color: var(--dk-muted); }
.ws-drop__text { display: grid; flex: 1; min-width: 0; line-height: 1.3; }
.ws-drop__text b { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-size: var(--dk-text-15); }
.ws-drop__text small { color: var(--dk-muted); font-size: var(--dk-text-12); }
.ws-drop .ws-source__x { margin-right: 0; }
/* The saved-voice line under the TTS prompt. */
.ws-voicebar { display: flex; flex-wrap: wrap; align-items: center; gap: var(--dk-space-3); }
.ws-voicebar__who { display: inline-flex; align-items: center; gap: var(--dk-space-2); font-size: var(--dk-text-13); }
.ws-voicebar__who small { color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
.ws-avatar { display: grid; place-items: center; width: 26px; height: 26px; border-radius: 50%; background: var(--dk-inset); box-shadow: inset 0 0 0 1px var(--dk-edge); font: var(--dk-weight-semibold) 10px / 1 var(--dk-font-body); color: var(--dk-text); }
.ws-hint--error { color: var(--dk-error-text); }
/* The Advanced fold. */
.ws-fold { border-top: 1px solid var(--dk-edge); }
.ws-fold__bar {
display: flex;
align-items: center;
gap: var(--dk-space-3);
width: 100%;
min-height: 44px;
padding: 0;
border: 0;
background: none;
color: var(--dk-text);
font: inherit;
text-align: left;
cursor: pointer;
}
.ws-fold__bar b { font: var(--dk-weight-semibold) var(--dk-text-15) / 1 var(--dk-font-body); }
.ws-fold__sum { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
.ws-fold__bar .lai-icon { margin-left: auto; color: var(--dk-muted); }
.ws-fold__bar:hover .lai-icon { color: var(--dk-text); }
.ws-fold__bar:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; border-radius: var(--dk-radius-1); }
.ws-fold__body { display: grid; gap: var(--dk-space-3); padding: var(--dk-space-1) 0 var(--dk-space-3); }
/* The foot: estimate, what changed, the reason, the action. */
.ws-foot { display: flex; flex-wrap: wrap; align-items: center; gap: var(--dk-space-3) var(--dk-space-4); padding-top: var(--dk-space-1); }
.ws-estimate { display: inline-flex; align-items: center; gap: var(--dk-space-2); min-height: 20px; color: var(--dk-muted); font: 500 12px / 1.3 var(--dk-font-mono); }
.ws-estimate:empty { display: none; }
.ws-estimate[data-fit="spill"] { color: var(--dk-warn-text); }
.ws-estimate[data-fit="over"] { color: var(--dk-error-text); }
.ws-changes { display: inline-flex; flex-wrap: wrap; align-items: center; gap: 6px; color: var(--dk-muted); font-size: var(--dk-text-12); }
.ws-changes code {
padding: 3px 8px;
border-radius: var(--dk-radius-1);
background: var(--dk-accent-wash);
color: var(--dk-accent-text);
font: 500 11.5px / 1.2 var(--dk-font-mono);
}
.ws-go { display: inline-flex; align-items: center; gap: var(--dk-space-3); margin-left: auto; flex-wrap: wrap; justify-content: flex-end; }
.ws-go__why { color: var(--dk-muted); font-size: var(--dk-text-13); }
/* --- Run area -------------------------------------------------------------- */
.ws-run { display: grid; gap: var(--dk-space-4); min-width: 0; background: none; border: 0; padding: 0; min-height: 0; border-radius: 0; }
.ws-empty {
display: flex;
width: 100%;
align-items: center;
gap: var(--dk-space-3);
padding: var(--dk-space-4) var(--dk-space-5);
border: 1.5px dashed var(--dk-control-edge);
border-radius: var(--dk-radius-4);
color: var(--dk-muted);
font-size: var(--dk-text-13);
}
.ws-empty p { margin: 0; }
.ws-empty .lai-icon { font-size: 20px; }
.ws-job {
display: grid;
grid-template-columns: minmax(0, 1fr) minmax(0, 260px);
gap: var(--dk-space-5);
align-items: center;
padding: var(--dk-space-5);
border-radius: var(--dk-radius-4);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
}
.ws-job:not(:has(.ws-job__tiles)) { grid-template-columns: 1fr; }
.ws-job__text { display: grid; gap: var(--dk-space-3); }
.ws-job__title { display: flex; align-items: center; gap: var(--dk-space-2); margin: 0; font: var(--dk-weight-semibold) var(--dk-text-17) / 1.2 var(--dk-font-display); }
.ws-job__time { margin-left: auto; color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
.ws-job__detail { margin: 0; color: var(--dk-muted); font: 500 12px / 1.4 var(--dk-font-mono); overflow-wrap: anywhere; }
.ws-job__tiles { display: grid; gap: var(--dk-space-2); }
.ws-job__tiles--n1 { grid-template-columns: 1fr; }
.ws-job__tiles--n2, .ws-job__tiles--n3, .ws-job__tiles--n4 { grid-template-columns: repeat(2, 1fr); }
.ws-job__tile { display: block; width: 100%; height: auto; aspect-ratio: var(--ratio, 1); border-radius: var(--dk-radius-2); }
@media (prefers-reduced-motion: reduce) {
.ws-job .dk-progress[data-indeterminate] .dk-progress-bar { animation: none; width: 100%; opacity: .45; }
}
.ws-failed {
display: flex;
gap: var(--dk-space-3);
padding: var(--dk-space-4) var(--dk-space-5);
border-radius: var(--dk-radius-4);
background: var(--dk-error-wash);
box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--dk-error) 40%, transparent);
color: var(--dk-text);
}
.ws-failed > .lai-icon { flex: none; margin-top: 2px; font-size: 20px; color: var(--dk-error-text); }
.ws-failed__body { display: grid; gap: var(--dk-space-2); min-width: 0; }
.ws-failed h3 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-17) / 1.3 var(--dk-font-display); color: var(--dk-error-text); }
.ws-failed p { margin: 0; max-width: 72ch; font-size: var(--dk-text-13); line-height: 1.5; overflow-wrap: anywhere; }
.ws-failed__note { color: var(--dk-muted); }
.ws-failed__row { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); margin-top: var(--dk-space-1); }
/* --- Result ----------------------------------------------------------------- */
.ws-result {
position: relative;
min-width: 0;
overflow: visible;
border-radius: var(--dk-radius-4);
background: var(--dk-card);
box-shadow: inset 0 0 0 1px var(--dk-edge);
}
.ws-result__head { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); padding: var(--dk-space-3) var(--dk-space-4); border-bottom: 1px solid var(--dk-edge); }
.ws-result__who { min-width: 0; flex: 1 1 220px; }
.ws-result__who h2 { margin: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); }
.ws-result__who p { margin: 2px 0 0; color: var(--dk-muted); font: 500 12px / 1.3 var(--dk-font-mono); }
.ws-result__tools { display: inline-flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-2); }
.ws-tool {
display: grid;
place-items: center;
width: 32px;
height: 32px;
padding: 0;
border: 0;
border-radius: var(--dk-radius-2);
background: transparent;
color: var(--dk-text);
font-size: 18px;
cursor: pointer;
text-decoration: none;
transition: background-color var(--dk-dur-1) ease, transform var(--dk-dur-1) var(--dk-ease-out);
}
.ws-tool:hover { background: var(--dk-hover); }
.ws-tool:active { transform: scale(var(--dk-press-scale)); }
.ws-tool:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 1px; }
.ws-tool .studio-star__glyph { width: 18px; height: 18px; fill: none; stroke: currentColor; stroke-width: 1.8; stroke-linejoin: round; }
.ws-tool[aria-pressed="true"] .studio-star__glyph { fill: currentColor; }
.ws-tool[aria-pressed="true"] { color: var(--dk-warn-text); }
.ws-result__view { background: var(--dk-inset); padding: var(--dk-space-4); display: grid; justify-items: center; align-content: start; gap: var(--dk-space-3); min-width: 0; }
.ws-result__view > * { max-width: 100%; }
.ws-result__view img, .ws-result__view video { max-width: 100%; max-height: min(70vh, 640px); border-radius: var(--dk-radius-2); }
.ws-result__meta { display: flex; flex-wrap: wrap; gap: var(--dk-space-1) var(--dk-space-4); padding: var(--dk-space-3) var(--dk-space-4); border-top: 1px solid var(--dk-edge); color: var(--dk-muted); font: 500 12px / 1.4 var(--dk-font-mono); }
.ws-result__meta:empty { display: none; }
.ws-result__meta a { color: var(--dk-accent-text); }
.ws-grid-images { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: var(--dk-space-3); width: 100%; }
.ws-grid-images--one { grid-template-columns: minmax(0, 560px); justify-content: center; }
.ws-grid-images .media-result-thumb { border-radius: var(--dk-radius-2); }
.ws-grid-images .media-result-thumb img { width: 100%; max-height: min(70vh, 640px); object-fit: contain; background: var(--dk-card); }
/* Audio viewers keep the player's own waveform and controls. */
.ws-audio { display: grid; gap: var(--dk-space-3); width: 100%; }
.ws-audio .audio-waveform-player { background: var(--dk-card); border-radius: var(--dk-radius-3); box-shadow: inset 0 0 0 1px var(--dk-edge); padding: var(--dk-space-3); }
.ws-quote { margin: 0; max-width: 70ch; color: var(--dk-text); font-size: var(--dk-text-15); line-height: 1.5; justify-self: start; overflow-wrap: anywhere; }
/* Use in menu. */
.ws-menu { min-width: 280px; z-index: 30; }
.ws-menu__none { margin: 0; padding: var(--dk-space-3); color: var(--dk-muted); font-size: var(--dk-text-13); }
.ws-menu__item { height: auto; min-height: 48px; padding: var(--dk-space-2) var(--dk-space-3); align-items: flex-start; }
.ws-menu__item .lai-icon { margin-top: 2px; }
.ws-menu__text { display: grid; gap: 3px; line-height: 1.25; }
.ws-menu__text b { font-weight: var(--dk-weight-semibold); font-size: var(--dk-text-13); }
.ws-menu__text small { color: var(--dk-muted); font-size: var(--dk-text-12); }
.ws-menu__item[data-state="missing"] small { color: var(--dk-warn-text); }
.ws-menu__item:disabled { opacity: 1; pointer-events: auto; cursor: not-allowed; }
.ws-menu__item:disabled b, .ws-menu__item:disabled .lai-icon { opacity: .55; }
/* --- Strip ------------------------------------------------------------------- */
.ws-strip__head { display: flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); }
.ws-strip__head h2 { margin: 0; font: var(--dk-weight-semibold) var(--dk-text-17) / 1.2 var(--dk-font-display); letter-spacing: -.01em; }
.ws-strip__count { color: var(--dk-muted); font: 500 12px / 1 var(--dk-font-mono); }
.ws-strip__tools { display: inline-flex; align-items: center; flex-wrap: wrap; gap: var(--dk-space-3); margin-left: auto; }
.ws-strip__filters { margin: 0; }
.ws-strip__clear {
display: grid;
place-items: center;
width: 30px;
height: 30px;
border: 0;
border-radius: var(--dk-radius-1);
background: transparent;
color: var(--dk-muted);
cursor: pointer;
}
.ws-strip__clear:hover { background: var(--dk-hover); color: var(--dk-text); }
.ws-strip__empty { margin: var(--dk-space-3) 0 0; color: var(--dk-muted); font-size: var(--dk-text-13); }
.ws-strip__list { display: flex; flex-wrap: wrap; gap: var(--dk-space-3); margin: var(--dk-space-3) 0 0; padding: 0; list-style: none; }
.ws-tile { position: relative; width: 132px; }
.ws-tile__open {
display: grid;
width: 100%;
gap: 6px;
padding: 0;
border: 0;
background: none;
color: inherit;
font: inherit;
text-align: left;
cursor: pointer;
}
.ws-tile__open .studio-thumb { height: 98px; border-radius: var(--dk-radius-2); box-shadow: inset 0 0 0 1px var(--dk-edge); transition: box-shadow var(--dk-dur-1) ease; }
.ws-tile__open:hover .studio-thumb { box-shadow: inset 0 0 0 1px var(--dk-control-edge); }
.ws-tile__open[aria-pressed="true"] .studio-thumb { box-shadow: inset 0 0 0 2px var(--dk-text); }
.ws-tile__open:focus-visible { outline: 2px solid var(--dk-accent); outline-offset: 2px; border-radius: var(--dk-radius-2); }
.ws-tile[data-active] .ws-tile__title { font-weight: var(--dk-weight-semibold); }
.ws-tile__cap { display: grid; gap: 2px; min-width: 0; }
.ws-tile__title { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-size: var(--dk-text-12); color: var(--dk-text); }
.ws-tile__cap small { color: var(--dk-muted); font: 500 11px / 1.2 var(--dk-font-mono); }
.ws-tile__star {
position: absolute;
left: 6px;
top: 6px;
display: grid;
place-items: center;
width: 24px;
height: 24px;
border-radius: var(--dk-radius-1);
background: rgb(10 16 15 / .72);
color: #f1f5f4;
pointer-events: none;
}
.ws-tile__star svg { width: 13px; height: 13px; fill: currentColor; stroke: currentColor; stroke-width: 1.5; stroke-linejoin: round; }
.ws-tile__del {
position: absolute;
right: 6px;
top: 6px;
display: grid;
place-items: center;
width: 24px;
height: 24px;
padding: 0;
border: 0;
border-radius: var(--dk-radius-1);
background: rgb(10 16 15 / .72);
color: #f1f5f4;
font-size: 14px;
cursor: pointer;
opacity: 0;
transition: opacity var(--dk-dur-1) ease;
}
.ws-tile:hover .ws-tile__del, .ws-tile__del:focus-visible { opacity: 1; }
@media (hover: none) { .ws-tile__del { opacity: 1; } }
/* The studio-thumb inside a tile fills the tile width. */
.ws-tile .studio-thumb--compact { width: 100%; }
/* --- Phone ------------------------------------------------------------------- */
@media (max-width: 640px) {
.ws { gap: var(--dk-space-4); }
.ws-compose { padding: var(--dk-space-4); }
.ws-job { grid-template-columns: 1fr; padding: var(--dk-space-4); }
.ws-go { margin-left: 0; width: 100%; justify-content: space-between; }
.ws-go__btn { flex: 1 1 auto; justify-content: center; }
.ws-strip__tools { margin-left: 0; }
.ws-tile { width: calc(50% - var(--dk-space-3) / 2); }
.ws-result__tools { width: 100%; }
.ws-result__tools .dk-anchor { position: static; }
.ws-menu { top: auto; left: var(--dk-space-3); right: var(--dk-space-3); min-width: 0; max-width: none; }
}
/* --- Inputs of the audio types ------------------------------------------------ */
.ws-inputs { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: var(--dk-space-4); align-items: start; }
.ws-inputs--one { grid-template-columns: minmax(0, 1fr); }
.ws-compose .ws-inputs .form-group { margin: 0; }
.ws-compose .ws-inputs .form-label,
.ws-compose .ws-inputs .biometrics-mediainput__label { font: var(--dk-weight-semibold) 11px / 1.2 var(--dk-font-body); letter-spacing: .07em; text-transform: uppercase; color: var(--dk-muted); }
.ws-compose .ws-inputs .form-help { color: var(--dk-muted); font-size: var(--dk-text-12); }
.ws-compose .audio-transform-echo { margin-top: var(--dk-space-1); }
.ws-chip-group { display: contents; }
.ws-optin { display: grid; gap: var(--dk-space-1); }
.ws-optin .ws-hint { max-width: 72ch; }
.ws-tool--text { width: auto; padding: 0 var(--dk-space-2); font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-mono); }
/* --- Viewers ------------------------------------------------------------------- */
.ws-viewer { display: grid; gap: var(--dk-space-3); width: 100%; min-width: 0; }
.ws-result__view .glb-viewer, .ws-viewer .glb-viewer { width: 100%; }
.ws-viewer .threed-remesh-controls { background: var(--dk-card); border-radius: var(--dk-radius-3); box-shadow: inset 0 0 0 1px var(--dk-edge); border: 0; }
.ws-audio .audio-spectrogram__canvas-wrap--empty { height: 140px; }
/* Diarization: timeline, talk time, segments. */
.ws-diar { display: grid; gap: var(--dk-space-3); width: 100%; min-width: 0; justify-items: stretch; }
.ws-diar h3 { margin: var(--dk-space-2) 0 0; font: var(--dk-weight-semibold) var(--dk-text-15) / 1.3 var(--dk-font-display); }
.ws-diar__head { display: flex; align-items: center; gap: var(--dk-space-3); }
.ws-diar__audio { width: 100%; }
.ws-timeline { display: grid; gap: var(--dk-space-2); padding: var(--dk-space-3); border-radius: var(--dk-radius-3); background: var(--dk-card); box-shadow: inset 0 0 0 1px var(--dk-edge); }
.ws-timeline__axis, .ws-timeline__lane { display: grid; grid-template-columns: 132px minmax(0, 1fr); gap: var(--dk-space-3); align-items: center; }
.ws-timeline__axis { position: relative; height: 18px; display: block; margin-left: calc(132px + var(--dk-space-3)); }
.ws-timeline__axis span { position: absolute; left: var(--at); transform: translateX(-50%); color: var(--dk-muted); font: 500 11px / 1 var(--dk-font-mono); }
.ws-timeline__axis span:first-child { transform: none; }
.ws-timeline__axis span:last-child { transform: translateX(-100%); }
.ws-timeline__who { display: inline-flex; align-items: center; gap: var(--dk-space-2); min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font: var(--dk-weight-semibold) 12px / 1 var(--dk-font-mono); }
.ws-timeline__who i, .ws-speaker h4 i, .ws-segments li strong::before {
flex: none;
display: inline-block;
width: 10px;
height: 10px;
border-radius: 3px;
background: var(--sp, var(--dk-series-1));
}
.ws-timeline__track { position: relative; height: 22px; border-radius: var(--dk-radius-1); background: var(--dk-inset); box-shadow: inset 0 0 0 1px var(--dk-edge); }
.ws-timeline__track b { position: absolute; left: var(--from); width: var(--len); top: 0; bottom: 0; border-radius: 3px; background: var(--sp, var(--dk-series-1)); }
[data-speaker="0"] { --sp: var(--dk-series-1); }
[data-speaker="1"] { --sp: var(--dk-series-2); }
[data-speaker="2"] { --sp: var(--dk-series-3); }
[data-speaker="3"] { --sp: var(--dk-series-4); }
[data-speaker="4"] { --sp: var(--dk-series-5); }
[data-speaker="5"] { --sp: var(--dk-series-6); }
.ws-speakers { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: var(--dk-space-3); margin: 0; padding: 0; list-style: none; }
.ws-speaker { display: grid; gap: var(--dk-space-2); align-content: start; padding: var(--dk-space-3); border-radius: var(--dk-radius-3); background: var(--dk-card); box-shadow: inset 0 0 0 1px var(--dk-edge); }
.ws-speaker h4 { display: flex; align-items: center; gap: var(--dk-space-2); margin: 0; font: var(--dk-weight-semibold) var(--dk-text-13) / 1.2 var(--dk-font-mono); overflow-wrap: anywhere; }
.ws-speaker p { margin: 0; color: var(--dk-text); font-size: var(--dk-text-13); line-height: 1.4; }
.ws-speaker .ws-speaker__talk { color: var(--dk-muted); font: 500 12px / 1.3 var(--dk-font-mono); }
.ws-speaker__bar { display: block; height: 4px; border-radius: 2px; background: var(--dk-inset); box-shadow: inset 0 0 0 1px var(--dk-edge); overflow: hidden; }
.ws-speaker__bar b { display: block; width: var(--w); height: 100%; background: var(--sp, var(--dk-series-1)); }
.ws-segments { display: grid; margin: 0; padding: 0; list-style: none; border-top: 1px solid var(--dk-edge); }
.ws-segments li {
display: grid;
grid-template-columns: 52px minmax(0, 150px) minmax(0, 1fr);
grid-template-areas: "time who range" ". text text";
gap: 2px var(--dk-space-3);
align-items: baseline;
padding: var(--dk-space-2) 0;
border-bottom: 1px solid var(--dk-edge);
font-size: var(--dk-text-13);
}
.ws-segments__time { grid-area: time; color: var(--dk-muted); font: 500 12px / 1.3 var(--dk-font-mono); }
.ws-segments li strong { grid-area: who; display: inline-flex; align-items: center; gap: 6px; min-width: 0; font: var(--dk-weight-semibold) 12px / 1.3 var(--dk-font-mono); overflow-wrap: anywhere; }
.ws-segments__range { grid-area: range; color: var(--dk-muted); font: 500 11px / 1.3 var(--dk-font-mono); justify-self: end; }
.ws-segments li p { grid-area: text; margin: 0; color: var(--dk-text); font-size: var(--dk-text-15); line-height: 1.5; }
@media (max-width: 640px) {
.ws-timeline__axis span:nth-child(even) { display: none; }
.ws-inputs { grid-template-columns: minmax(0, 1fr); }
.ws-timeline__axis { margin-left: calc(104px + var(--dk-space-3)); }
.ws-timeline__axis, .ws-timeline__lane { grid-template-columns: 104px minmax(0, 1fr); }
.ws-segments li { grid-template-columns: 44px minmax(0, 1fr); grid-template-areas: "time who" ". range" ". text"; }
.ws-segments__range { justify-self: start; }
}
/* The kit has no file input; the browser's own button is quieted to match. */
.ws-compose input[type="file"] {
max-width: 100%;
padding: var(--dk-space-2);
border: 1px solid var(--dk-control-edge);
border-radius: var(--dk-radius-2);
background: var(--dk-card);
color: var(--dk-text);
font-size: var(--dk-text-13);
}
.ws-compose input[type="file"]::file-selector-button {
margin-right: var(--dk-space-3);
padding: 6px var(--dk-space-3);
border: 0;
border-radius: var(--dk-radius-1);
background: var(--dk-inset);
box-shadow: inset 0 0 0 1px var(--dk-edge);
color: var(--dk-text);
font: var(--dk-weight-semibold) var(--dk-text-13) / 1 var(--dk-font-body);
cursor: pointer;
}
.ws-strip__empty { text-align: left; }
addons/studio/ui/src/hooks/use3DHistory.js
+154
View File
@@ -0,0 +1,154 @@
import { useState, useEffect, useCallback, useMemo } from 'react'
import { generateId } from '../utils/format'
// use3DHistory — IndexedDB-backed history of 3D generations. Unlike
// useMediaHistory (localStorage, URL-only), entries here carry the generated
// GLB as a Blob: GLBs are multi-megabyte binaries the server may eventually
// clean up, and IndexedDB is the only browser store that handles blobs of
// that size. Blobs read back from IndexedDB are lazy handles (the bytes are
// not materialized into JS memory until used), so a single store is fine.
//
// Entry: { id, createdAt, name, model,
// params: { seed, steps, textureSteps, guidance, quality, background },
// inputThumb, // small dataURL of the conditioning image
// glb, // Blob
// parentId, edge, label } // optional: the result it was made from
const DB_NAME = 'localai-3d-history'
const DB_VERSION = 1
const STORE = 'generations'
const MAX_ENTRIES = 20
const historyListeners = new Set()
let sessionEntries = []
async function refreshOtherHooks(source) {
await Promise.all([...historyListeners]
.filter(listener => listener !== source)
.map(listener => listener()))
}
function openDb() {
return new Promise((resolve, reject) => {
const req = indexedDB.open(DB_NAME, DB_VERSION)
req.onupgradeneeded = () => {
const db = req.result
if (!db.objectStoreNames.contains(STORE)) {
db.createObjectStore(STORE, { keyPath: 'id' }).createIndex('createdAt', 'createdAt')
}
}
req.onsuccess = () => resolve(req.result)
req.onerror = () => reject(req.error)
})
}
const txDone = (tx) => new Promise((resolve, reject) => {
tx.oncomplete = resolve
tx.onerror = () => reject(tx.error)
tx.onabort = () => reject(tx.error)
})
async function withStore(mode, fn) {
const db = await openDb()
try {
const tx = db.transaction(STORE, mode)
const result = fn(tx.objectStore(STORE))
await txDone(tx)
return result
} finally {
db.close()
}
}
async function idbGetAll() {
const req = await withStore('readonly', (store) => store.getAll())
return (req.result || []).sort((a, b) => b.createdAt - a.createdAt)
}
// Insert + keep-newest-N eviction in one transaction so a crash between the
// two can't leave the store unbounded.
async function idbPutAndEvict(entry) {
await withStore('readwrite', (store) => {
store.put(entry)
// getAllKeys on the createdAt index yields primary keys oldest-first.
const keysReq = store.index('createdAt').getAllKeys()
keysReq.onsuccess = () => {
const excess = keysReq.result.length - MAX_ENTRIES
for (let i = 0; i < excess; i++) store.delete(keysReq.result[i])
}
})
}
const idbDelete = (id) => withStore('readwrite', (store) => store.delete(id))
const idbClear = () => withStore('readwrite', (store) => store.clear())
export function use3DHistory() {
const [entries, setEntries] = useState([])
const [selectedId, setSelectedId] = useState(null)
const refresh = useCallback(async () => {
try {
sessionEntries = await idbGetAll()
setEntries(sessionEntries)
} catch {
// IndexedDB unavailable (private mode etc.) — degrade to session-only.
setEntries(sessionEntries)
}
}, [])
useEffect(() => {
historyListeners.add(refresh)
refresh()
return () => { historyListeners.delete(refresh) }
}, [refresh])
const addEntry = useCallback(async ({ model, params, inputThumb, glb, name, inputs, operation, outputType, parentId, edge, label }) => {
const entry = {
id: generateId(), createdAt: Date.now(), model, params, inputThumb, glb, name, inputs, operation, outputType,
...(parentId ? { parentId, edge: edge || 'to-3d' } : {}),
...(label ? { label: String(label).slice(0, 2000) } : {}),
}
try {
await idbPutAndEvict(entry)
await refresh()
} catch {
sessionEntries = [entry, ...sessionEntries.filter(e => e.id !== entry.id)].slice(0, MAX_ENTRIES)
setEntries(sessionEntries)
}
await refreshOtherHooks(refresh)
return entry
}, [refresh])
const deleteEntry = useCallback(async (id) => {
setSelectedId((prev) => (prev === id ? null : prev))
try {
await idbDelete(id)
await refresh()
} catch {
sessionEntries = sessionEntries.filter((e) => e.id !== id)
setEntries(sessionEntries)
}
await refreshOtherHooks(refresh)
}, [refresh])
const clearAll = useCallback(async () => {
setSelectedId(null)
try {
await idbClear()
} catch { /* session-only history is cleared below */ }
sessionEntries = []
setEntries([])
await refreshOtherHooks(refresh)
}, [refresh])
// Toggles: clicking the selected entry deselects it (back to latest result).
const selectEntry = useCallback((id) => {
setSelectedId((prev) => (prev === id ? null : id))
}, [])
const selectedEntry = useMemo(
() => entries.find((e) => e.id === selectedId) || null,
[entries, selectedId],
)
return { entries, addEntry, deleteEntry, clearAll, selectEntry, selectedId, selectedEntry }
}
addons/studio/ui/src/hooks/useMediaHistory.js
+133
View File
@@ -0,0 +1,133 @@
import { useState, useCallback, useRef, useEffect, useMemo } from 'react'
import { generateId } from '../utils/format'
import { MEDIA_STORAGE_KEYS as STORAGE_KEYS, FAVOURITES_KEY, clearAllMediaHistory } from '../utils/mediaHistoryStore'
export { FAVOURITES_KEY, clearAllMediaHistory }
const SAVE_DEBOUNCE_MS = 500
const MAX_ENTRIES = 100
// A prompt is a sentence or a paragraph. This keeps one runaway paste from
// filling the quota that every other entry shares.
const MAX_PROMPT_CHARS = 2000
// Read every store at once, without mounting a hook per media type.
//
// The Studio overview reads across all of them and writes to none, while
// useMediaHistory() carries save timers and selection state it would have no
// use for. Note there is no 'threed' store here: 3D history lives in IndexedDB
// rather than localStorage because its entries carry multi-megabyte GLB blobs,
// so callers wanting 3D read use3DHistory alongside this.
export function readAllMediaHistory() {
const all = {}
for (const [mediaType, key] of Object.entries(STORAGE_KEYS)) {
all[mediaType] = loadEntries(key)
}
return all
}
function loadEntries(key) {
try {
const stored = localStorage.getItem(key)
if (stored) {
const data = JSON.parse(stored)
if (Array.isArray(data)) return data
}
} catch (_e) {
localStorage.removeItem(key)
}
return []
}
function saveEntries(key, entries) {
try {
localStorage.setItem(key, JSON.stringify(entries))
} catch (err) {
if (err.name === 'QuotaExceededError' || err.code === 22) {
console.warn('localStorage quota exceeded for media history')
}
}
}
export function useMediaHistory(mediaType) {
const storageKey = STORAGE_KEYS[mediaType]
const [entries, setEntries] = useState(() => loadEntries(storageKey))
const [selectedId, setSelectedId] = useState(null)
const saveTimer = useRef(null)
const dirty = useRef(false)
const entriesRef = useRef(entries)
entriesRef.current = entries
// Debounced save — only when dirty
useEffect(() => {
if (!dirty.current) return
clearTimeout(saveTimer.current)
saveTimer.current = setTimeout(() => {
saveEntries(storageKey, entries)
dirty.current = false
}, SAVE_DEBOUNCE_MS)
return () => clearTimeout(saveTimer.current)
}, [entries, storageKey])
// Flush pending save on unmount
useEffect(() => {
return () => {
clearTimeout(saveTimer.current)
if (dirty.current) {
saveEntries(storageKey, entriesRef.current)
}
}
}, [storageKey])
// parentId and edge link a result to the one it was made from (a hand-off
// from another Studio workspace); they are written only when there is one.
const addEntry = useCallback(({ prompt, model, params, results, parentId, edge }) => {
const entry = {
id: generateId(),
prompt: typeof prompt === 'string' ? prompt.slice(0, MAX_PROMPT_CHARS) : prompt,
model,
params: params || {},
results: results || [],
createdAt: Date.now(),
...(parentId ? { parentId, edge: edge || 'take' } : {}),
}
dirty.current = true
setEntries(prev => {
const updated = [entry, ...prev]
if (updated.length > MAX_ENTRIES) updated.length = MAX_ENTRIES
return updated
})
return entry.id
}, [])
const deleteEntry = useCallback((id) => {
dirty.current = true
setEntries(prev => prev.filter(e => e.id !== id))
setSelectedId(prev => prev === id ? null : prev)
}, [])
const clearAll = useCallback(() => {
dirty.current = true
setEntries([])
setSelectedId(null)
}, [])
const selectEntry = useCallback((id) => {
setSelectedId(prev => prev === id ? null : id)
}, [])
const selectedEntry = useMemo(
() => entries.find(e => e.id === selectedId) || null,
[entries, selectedId]
)
const historyProps = useMemo(() => ({
entries,
selectedId,
onSelect: selectEntry,
onDelete: deleteEntry,
onClearAll: clearAll,
mediaType,
}), [entries, selectedId, selectEntry, deleteEntry, clearAll, mediaType])
return { addEntry, selectEntry, selectedEntry, historyProps }
}
addons/studio/ui/src/hooks/useModelSuggestion.js
+102
View File
@@ -0,0 +1,102 @@
import { useEffect, useState } from 'react'
import { modelsApi } from '../utils/api'
import { TYPE_INFO } from '../utils/studioWork'
// One gallery model to suggest for a Studio type that has none installed.
//
// The gallery has no "best for images" flag, so this reads what it does have:
// models carrying the type's tag, in the gallery's own order, not yet installed.
// It asks for the size of the first few and suggests the smallest download whose
// size is known; if no size is known it suggests the first. Whatever the server
// does not report stays null and the note says it is unknown, never a number
// made up to fill the line.
//
// Results are kept per tag for the session, so picking the chip again, or
// switching between two types, does not ask twice.
const cache = new Map()
const CANDIDATES = 4
const DEFAULT_CTX = 4096
async function lookup(tag) {
const data = await modelsApi.list({ tag, items: 8, page: 1 })
const candidates = (data?.models || []).filter(m => !m.installed).slice(0, CANDIDATES)
if (candidates.length === 0) return null
const sized = await Promise.all(candidates.map(async (m) => {
const name = m.name || m.id
try {
const e = await modelsApi.estimate(name, [DEFAULT_CTX])
const ctx = e?.estimates?.[String(DEFAULT_CTX)]
return {
name,
description: m.description || '',
sizeBytes: e?.sizeBytes ?? null,
sizeDisplay: e?.sizeDisplay ?? null,
vramBytes: ctx?.vramBytes ?? null,
vramDisplay: ctx?.vramDisplay ?? null,
}
} catch {
return { name, description: m.description || '', sizeBytes: null, sizeDisplay: null, vramBytes: null, vramDisplay: null }
}
}))
const known = sized.filter(c => c.sizeBytes > 0).sort((a, b) => a.sizeBytes - b.sizeBytes)
return known[0] || sized[0]
}
// state: 'idle' (not asked), 'loading', 'ready' (suggestion set), 'none' (the
// gallery has nothing for this type), 'error' (the gallery is out of reach).
export function useModelSuggestion(type, enabled) {
const tag = TYPE_INFO[type]?.tag
const [state, setState] = useState({ state: 'idle', suggestion: null })
useEffect(() => {
if (!enabled || !tag) { setState({ state: 'idle', suggestion: null }); return undefined }
if (cache.has(tag)) {
const hit = cache.get(tag)
setState({ state: hit ? 'ready' : 'none', suggestion: hit })
return undefined
}
let cancelled = false
setState({ state: 'loading', suggestion: null })
lookup(tag)
.then((hit) => {
cache.set(tag, hit)
if (!cancelled) setState({ state: hit ? 'ready' : 'none', suggestion: hit })
})
.catch(() => { if (!cancelled) setState({ state: 'error', suggestion: null }) })
return () => { cancelled = true }
}, [tag, enabled])
return state
}
// The memory an installed model needs, when the gallery can say. Installed
// models carry no size in the capabilities list, so this asks the same estimate
// endpoint by name; a model that is not a gallery entry answers with nothing and
// the chip simply shows no fit.
const needCache = new Map()
export function useModelNeed(modelId) {
const [need, setNeed] = useState(() => needCache.get(modelId) ?? null)
useEffect(() => {
if (!modelId) { setNeed(null); return undefined }
if (needCache.has(modelId)) { setNeed(needCache.get(modelId)); return undefined }
let cancelled = false
setNeed(null)
modelsApi.estimate(modelId, [DEFAULT_CTX])
.then((e) => {
const ctx = e?.estimates?.[String(DEFAULT_CTX)]
const value = ctx?.vramBytes > 0 ? ctx.vramBytes : null
needCache.set(modelId, value)
if (!cancelled) setNeed(value)
})
.catch(() => { needCache.set(modelId, null) })
return () => { cancelled = true }
}, [modelId])
return need
}
// Test hook: forget what was looked up.
export function resetSuggestionCache() {
cache.clear()
needCache.clear()
}
addons/studio/ui/src/hooks/useSoundExpansionModel.js
+46
View File
@@ -0,0 +1,46 @@
import { useCallback, useEffect, useState } from 'react'
import { useModels } from './useModels'
import { CAP_CHAT } from '../utils/capabilities'
export const BACKEND_DEFAULT = '__backend_default__'
const PREFERENCE_KEY = 'localai_sound_expansion_model'
function readPreference() {
try { return localStorage.getItem(PREFERENCE_KEY) || '' } catch { return '' }
}
function writePreference(value) {
try {
if (value) localStorage.setItem(PREFERENCE_KEY, value)
else localStorage.removeItem(PREFERENCE_KEY)
} catch { /* private mode or storage quota: keep the current session's choice */ }
}
// Sound must ask again when a selected model disappears, rather than silently
// switching the LLM. It also must not change the general chat preference.
export function useSoundExpansionModel() {
const { models, loading, error, refetch } = useModels(CAP_CHAT)
const [selected, setSelected] = useState(readPreference)
const valid = selected === BACKEND_DEFAULT || (!loading && !error && models.some(m => m.id === selected))
useEffect(() => {
if (!loading && !error && selected && !valid) {
setSelected('')
writePreference('')
}
}, [loading, error, selected, valid])
// Installation and capability edits may happen in another tab. Validate the
// saved choice against the refreshed list when the user returns to Sound.
useEffect(() => {
window.addEventListener('focus', refetch)
return () => window.removeEventListener('focus', refetch)
}, [refetch])
const choose = useCallback(value => {
setSelected(value)
writePreference(value)
}, [])
return { value: valid ? selected : '', choose, models, loading, error }
}
addons/studio/ui/src/hooks/useSpectrogram.js
+108
View File
@@ -0,0 +1,108 @@
import { useEffect, useState } from 'react'
import { getSharedAudioContext } from './useAudioPeaks'
import { fftRadix2 } from '../utils/fft'
import { fetchUrl } from '../utils/api'
// Hann windows are reused across frames and across clips, so cache one per
// size. The window tapers each frame to suppress spectral leakage (the
// vertical smearing you'd otherwise get from hard frame edges).
const windowCache = new Map()
function hann(n) {
let w = windowCache.get(n)
if (w) return w
w = new Float32Array(n)
for (let i = 0; i < n; i++) w[i] = 0.5 - 0.5 * Math.cos((2 * Math.PI * i) / (n - 1))
windowCache.set(n, w)
return w
}
const EMPTY = { spectrogram: null, frames: 0, bins: 0, maxFreq: 0, duration: 0, error: null, loading: false }
// useSpectrogram — decode an audio source (blob/data/http URL) and compute a
// magnitude STFT suitable for a spectrogram heatmap. Returns
// `{ spectrogram, frames, bins, maxFreq, duration, error, loading }` where
// `spectrogram` is a Float32Array of `frames * bins` values, row-major by
// frame, normalised so the dB floor maps to 0 and the loudest bin to 1.
// `bins` spans 0..Nyquist (`maxFreq`).
//
// fftSize/hop default to the LocalVQE frame geometry (512/256) so the picture
// lines up with how the model itself frames the audio. Long clips are
// strided down to at most `maxFrames` columns — the heatmap is only a few
// hundred px wide, so computing an FFT per native hop would be wasted work.
export default function useSpectrogram(
src,
{ fftSize = 512, hop = 256, maxFrames = 900, dbFloor = -90 } = {},
) {
const [state, setState] = useState(EMPTY)
useEffect(() => {
setState(EMPTY)
if (!src) return
let cancelled = false
setState((s) => ({ ...s, loading: true }))
async function run() {
try {
const resp = await fetchUrl(src)
const raw = await resp.arrayBuffer()
const ctx = getSharedAudioContext()
if (!ctx) throw new Error('Web Audio API not available')
const audio = await ctx.decodeAudioData(raw.slice(0))
if (cancelled) return
const data = audio.getChannelData(0)
const bins = fftSize >> 1
const win = hann(fftSize)
// Frame count, then a stride so we never run more than maxFrames FFTs.
const rawFrames = data.length >= fftSize ? 1 + Math.floor((data.length - fftSize) / hop) : 1
const stride = rawFrames > maxFrames ? Math.ceil(rawFrames / maxFrames) : 1
const frames = Math.ceil(rawFrames / stride)
const spec = new Float32Array(frames * bins)
const re = new Float64Array(fftSize)
const im = new Float64Array(fftSize)
let peakDb = dbFloor
for (let f = 0; f < frames; f++) {
const start = f * stride * hop
for (let i = 0; i < fftSize; i++) {
const s = start + i
re[i] = s < data.length ? data[s] * win[i] : 0
im[i] = 0
}
fftRadix2(re, im)
for (let b = 0; b < bins; b++) {
const mag = Math.hypot(re[b], im[b]) / fftSize
let db = mag > 0 ? 20 * Math.log10(mag) : dbFloor
if (db < dbFloor) db = dbFloor
spec[f * bins + b] = db
if (db > peakDb) peakDb = db
}
}
// Normalise dB into [0,1] against [dbFloor, peakDb].
const range = peakDb - dbFloor || 1
for (let i = 0; i < spec.length; i++) spec[i] = (spec[i] - dbFloor) / range
if (cancelled) return
setState({
spectrogram: spec,
frames,
bins,
maxFreq: audio.sampleRate / 2,
duration: audio.duration,
error: null,
loading: false,
})
} catch (e) {
if (!cancelled) setState((s) => ({ ...s, error: e?.message || 'Could not analyse audio', loading: false }))
}
}
run()
return () => { cancelled = true }
}, [src, fftSize, hop, maxFrames, dbFloor])
return state
}
addons/studio/ui/src/hooks/useStudioHandoff.js
+66
View File
@@ -0,0 +1,66 @@
import { useEffect, useMemo, useState } from 'react'
import { useSearchParams } from 'react-router-dom'
import { readAllMediaHistory } from './useMediaHistory'
import { fetchUrl } from '../utils/api'
import { collectWork, readHandoff, sourceUrlFor } from '../utils/studioWork'
// What the Studio front page hands to a workspace: a prompt, a model, a size, a
// count, and the id of the result the new one grows from. A workspace calls this
// once and seeds its own form from it; with no query string it returns empty
// values and the workspace behaves as it always has.
export function useStudioHandoff() {
const [params] = useSearchParams()
const key = params.toString()
// The query string is the only input, so its text is the dependency.
// eslint-disable-next-line react-hooks/exhaustive-deps
return useMemo(() => readHandoff(params), [key])
}
// The file a hand-off starts from, fetched from the server's own output URL.
//
// status 'none' no source was asked for
// 'loading' fetching
// 'ready' `blob` and `item` are set
// 'missing' the result is no longer in this browser's history
// 'error' the file could not be fetched (the server may have
// cleaned it up)
//
// Only the browser-storage lists are searched. A 3D result has no file URL, and
// nothing accepts a mesh as a source yet.
export function useHandoffSource(handoff, enabled = true) {
const { from, edge } = handoff
const [state, setState] = useState({ status: from && enabled ? 'loading' : 'none', blob: null, item: null })
useEffect(() => {
if (!from || !enabled) { setState({ status: 'none', blob: null, item: null }); return undefined }
const item = collectWork(readAllMediaHistory(), []).find(i => i.id === from) || null
const url = sourceUrlFor(item, edge)
if (!item || !url) { setState({ status: 'missing', blob: null, item }); return undefined }
let cancelled = false
setState({ status: 'loading', blob: null, item })
fetchUrl(url)
.then(res => { if (!res.ok) throw new Error(`HTTP ${res.status}`); return res.blob() })
.then(blob => { if (!cancelled) setState({ status: 'ready', blob, item }) })
.catch(() => { if (!cancelled) setState({ status: 'error', blob: null, item }) })
return () => { cancelled = true }
}, [from, edge, enabled])
return state
}
// A blob as the { base64, dataUrl, mime } shape the image inputs hold.
export function blobToImageInput(blob) {
return new Promise((resolve, reject) => {
const reader = new FileReader()
reader.onload = () => {
const dataUrl = String(reader.result)
resolve({ base64: dataUrl.split(',')[1] || '', dataUrl, mime: blob.type || 'image/png', source: 'upload' })
}
reader.onerror = reject
reader.readAsDataURL(blob)
})
}
export function blobToFile(blob, name) {
return new File([blob], name, { type: blob.type || 'application/octet-stream' })
}
addons/studio/ui/src/hooks/useStudioWork.js
+67
View File
@@ -0,0 +1,67 @@
import { useCallback, useEffect, useMemo, useState } from 'react'
import { clearAllMediaHistory, FAVOURITES_KEY, readAllMediaHistory, useMediaHistory } from './useMediaHistory'
import { use3DHistory } from './use3DHistory'
import { collectWork, pruneIds, toggleId } from '../utils/studioWork'
function readFavourites() {
try {
const stored = JSON.parse(localStorage.getItem(FAVOURITES_KEY) || '[]')
return Array.isArray(stored) ? stored.filter(id => typeof id === 'string') : []
} catch {
return []
}
}
function writeFavourites(ids) {
try {
if (ids.length === 0) localStorage.removeItem(FAVOURITES_KEY)
else localStorage.setItem(FAVOURITES_KEY, JSON.stringify(ids))
} catch { /* quota or private mode: the star simply does not stick */ }
}
// Everything the Studio front page shows as "your work": the entries each
// workspace stored, in one list, with favourites. The browser-storage lists are
// read once on mount, except images: inline upscaling writes through the same
// image history hook and merges its live entries. 3D arrives from IndexedDB.
//
// `ready` is false until 3D has answered, so the page can show a skeleton
// instead of flashing an empty state for work that is a moment away.
export function useStudioWork() {
const { addEntry: addImageEntry, historyProps: imageHistory } = useMediaHistory('image')
const clearImages = imageHistory.onClearAll
const [media, setMedia] = useState(() => readAllMediaHistory())
const [favourites, setFavourites] = useState(readFavourites)
const { entries: threeD, clearAll: clearThreeD } = use3DHistory()
const [ready, setReady] = useState(false)
// use3DHistory starts with [] and fills from IndexedDB; one tick later it has
// answered either way, which is all "ready" needs to mean.
useEffect(() => {
const id = setTimeout(() => setReady(true), 0)
return () => clearTimeout(id)
}, [])
useEffect(() => { if (threeD.length > 0) setReady(true) }, [threeD])
const items = useMemo(
() => collectWork({ ...media, image: imageHistory.entries }, threeD, { favourites }),
[media, imageHistory.entries, threeD, favourites],
)
const toggleFavourite = useCallback((id) => {
setFavourites(prev => {
const next = toggleId(pruneIds(prev, items), id)
writeFavourites(next)
return next
})
}, [items])
const clearHistory = useCallback(async () => {
clearImages()
clearAllMediaHistory()
setMedia(readAllMediaHistory())
setFavourites([])
await clearThreeD()
}, [clearThreeD, clearImages])
return { items, ready, addImageEntry, toggleFavourite, clearHistory }
}
addons/studio/ui/src/hooks/useWorkspace.js
+155
View File
@@ -0,0 +1,155 @@
import { useCallback, useMemo, useRef, useState } from 'react'
import { useNavigate } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import { useModels } from './useModels'
import { FAVOURITES_KEY } from './useMediaHistory'
import { readAllMediaHistory } from './useMediaHistory'
import {
CAP_3D, CAP_3D_ANIMATION, CAP_AUDIO_TRANSFORM, CAP_DIARIZATION, CAP_IMAGE, CAP_SOUND_GENERATION, CAP_TTS, CAP_VIDEO,
} from '../utils/capabilities'
import {
NEXT_STEPS, TYPE_INFO, TYPE_ORDER, canTake, collectWork, handoffPath, toggleId,
} from '../utils/studioWork'
// The model capabilities that make each Studio type usable. 3D accepts either
// the mesh or the animation flag, as the Studio tab dots do.
const TYPE_CAPABILITIES = {
images: [CAP_IMAGE],
video: [CAP_VIDEO],
threed: [CAP_3D, CAP_3D_ANIMATION],
tts: [CAP_TTS],
sound: [CAP_SOUND_GENERATION],
transform: [CAP_AUDIO_TRANSFORM],
diarization: [CAP_DIARIZATION],
}
function readFavourites() {
try {
const stored = JSON.parse(localStorage.getItem(FAVOURITES_KEY) || '[]')
return Array.isArray(stored) ? stored.filter(id => typeof id === 'string') : []
} catch {
return []
}
}
function writeFavourites(ids) {
try {
if (ids.length === 0) localStorage.removeItem(FAVOURITES_KEY)
else localStorage.setItem(FAVOURITES_KEY, JSON.stringify(ids))
} catch { /* quota or private mode: the star simply does not stick */ }
}
// Which Studio types have a model installed, from one unfiltered read of the
// capabilities list. `loading` is true until the first answer, so a workspace
// does not flash its "no model" note while the list is still on its way.
export function useInstalledTypes() {
const { models, loading, error, refetch } = useModels()
const byType = useMemo(() => {
const out = {}
for (const type of TYPE_ORDER) {
out[type] = models
.filter(m => !m.disabled && m.capabilities?.some(cap => TYPE_CAPABILITIES[type].includes(cap)))
.map(m => m.id)
}
return out
}, [models])
return { byType, loading, error, refetch }
}
// What the fold names when it is closed: the settings that hold a value, else
// what is inside.
export function foldSummary(set, all) {
return (set.length > 0 ? set : all).join(', ')
}
// How one value changed since the take it was started from, in words a person
// would use: a short value shows both sides, a long one just says it was edited.
export function describeChange(label, from, to) {
const a = String(from ?? '')
const b = String(to ?? '')
if (a.length > 24 || b.length > 24) return { key: label, text: label, long: true }
return { key: label, text: `${label} ${a || 'empty'} to ${b || 'empty'}` }
}
// Everything the shared workspace frame needs from one page: its recent
// results as work items, favourites, the "Use in" targets, the lineage link
// and the "Re-run with edits" comparison.
//
// type the Studio key ('images', 'video', ...)
// entries the page's own history entries, newest first (live, so a result
// made a moment ago is already there)
//
// Favourites are the same list the Studio front page keeps, so a star set here
// shows there.
export function useWorkspace({ type, entries }) {
const { t } = useTranslation('media')
const navigate = useNavigate()
const info = TYPE_INFO[type]
const [favourites, setFavourites] = useState(readFavourites)
const installed = useInstalledTypes()
const composeRef = useRef(null)
const [base, setBase] = useState(null)
const items = useMemo(
() => collectWork(info.media ? { [info.media]: entries } : {}, info.media ? [] : entries, { favourites }),
[entries, favourites, info.media],
)
const itemById = useCallback((id) => (id ? items.find(i => i.id === id) || null : null), [items])
const toggleFavourite = useCallback((id) => {
setFavourites((prev) => {
const next = toggleId(prev, id)
writeFavourites(next)
return next
})
}, [])
const openLineage = useCallback((item) => navigate(`/app/studio?work=${encodeURIComponent(item.id)}`), [navigate])
// The steps this kind of result can go on to, each marked with whether the
// destination can start from it and whether a model for it is installed.
const steps = useCallback((item) => (NEXT_STEPS[type] || []).map(step => ({
...step,
title: t(`studio.steps.${step.edge}.title`),
type: step.to,
missingModel: step.supported && (installed.byType[step.to]?.length || 0) === 0 && !installed.loading,
blocked: item ? !step.supported : true,
})), [type, t, installed.byType, installed.loading])
const sendTo = useCallback((item, step) => {
if (!item || !step?.supported) return
const textLike = TYPE_INFO[step.to].input === 'text'
const prompt = !textLike ? ''
: step.edge === 'variation' ? item.title
: t(`studio.steps.${step.edge}.template`, { defaultValue: '' })
navigate(handoffPath(step.to, { prompt, from: item.id, edge: step.edge }))
}, [navigate, t])
// Re-run with edits: the take's values are loaded into the form and kept as
// the comparison. `begin` is called by the page after it has set its own
// state; `changes` is computed from what the form holds now.
const begin = useCallback((values) => {
setBase(values)
composeRef.current?.scrollIntoView?.({ block: 'start', behavior: 'smooth' })
}, [])
const end = useCallback(() => setBase(null), [])
const changes = useCallback((current, labels) => {
if (!base) return []
return Object.keys(labels)
.filter(key => String(base[key] ?? '') !== String(current[key] ?? ''))
.map(key => ({ ...describeChange(labels[key], base[key], current[key]), field: key }))
}, [base])
return {
type, items, itemById, favourites, toggleFavourite, openLineage, steps, sendTo, canTake,
installed, composeRef, rerunning: !!base, begin, end, changes,
}
}
// The result a hand-off or a lineage link names, looked up across all stores,
// for the "from ..." line under a result.
export function findParent(item) {
if (!item?.parentId) return null
const all = collectWork(readAllMediaHistory(), [])
return all.find(i => i.id === item.parentId) || null
}
addons/studio/ui/src/i18n/media.de.json
+195
View File
@@ -0,0 +1,195 @@
{
"studio": {
"tabs": {
"images": "Bilder",
"video": "Video",
"tts": "TTS",
"sound": "Audio",
"transform": "Transform",
"overview": "Overview"
},
"overview": {
"eyebrow": "{{ready}} of {{total}} modalities ready",
"title": "Studio",
"subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
"canMake": "What you can make",
"running": "Running now",
"recent": "Recent outputs",
"noModel": "No model installed",
"install": "Install a model",
"ready": "Ready",
"seconds": "{{seconds}}s",
"describe": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "Image to mesh reconstruction",
"tts": "Text to speech using your voice library",
"sound": "Music and sound effects from a prompt",
"transform": "Separation, enhancement and voice conversion"
}
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
}
},
"image": {
"title": "Bildgenerierung",
"labels": {
"model": "Modell",
"prompt": "Prompt",
"promptPlaceholder": "Beschreiben Sie das Bild, das Sie generieren möchten...",
"negativePrompt": "Negativer Prompt",
"negativePromptPlaceholder": "Was vermieden werden soll...",
"size": "Größe",
"count": "Anzahl (1-4)",
"advanced": "Erweiterte Einstellungen",
"imageInputs": "Bildeingaben",
"steps": "Schritte",
"stepsPlaceholder": "20",
"seed": "Seed",
"seedPlaceholder": "Zufällig",
"sourceImage": "Ausgangsbild (img2img)",
"refImages": "Referenzbilder",
"refImagesAdded_one": "{{count}} Bild hinzugefügt",
"refImagesAdded_other": "{{count}} Bilder hinzugefügt"
},
"actions": {
"view": "View",
"generate": "Generieren",
"generating": "Generieren..."
},
"empty": "Generierte Bilder erscheinen hier",
"toasts": {
"noPrompt": "Bitte einen Prompt eingeben",
"noModel": "Bitte ein Modell auswählen",
"noResults": "Keine Bilder generiert"
}
},
"video": {
"title": "Videogenerierung",
"labels": {
"model": "Modell",
"prompt": "Prompt",
"promptPlaceholder": "Beschreiben Sie das Video, das Sie generieren möchten...",
"duration": "Dauer (s)",
"fps": "FPS",
"size": "Größe",
"advanced": "Erweiterte Einstellungen",
"seed": "Seed",
"seedPlaceholder": "Zufällig",
"frames": "Frames",
"referenceMedia": "Referenzmedien",
"startImage": "Startbild",
"endImage": "Endbild",
"avatarAudio": "Avatar-Audio"
},
"actions": {
"generate": "Video generieren",
"generating": "Generieren..."
},
"empty": "Generierte Videos erscheinen hier",
"toasts": {
"noPrompt": "Bitte einen Prompt eingeben",
"noModel": "Bitte ein Modell auswählen",
"noResults": "Keine Videos generiert"
}
},
"tts": {
"title": "Text zu Sprache",
"labels": {
"model": "Modell",
"voice": "Stimme",
"voicePlaceholder": "Optionale Stimm-ID",
"input": "Text",
"inputPlaceholder": "Text zum Synthetisieren eingeben...",
"instructions": "Anweisungen",
"instructionsPlaceholder": "Zum Beispiel: Langsam und warmherzig sprechen",
"instructionsHint": "Optionale Hinweise zur Sprechweise. Die Unterstützung hängt vom Backend ab."
},
"actions": {
"generate": "Audio generieren",
"generating": "Generieren..."
},
"empty": "Generiertes Audio erscheint hier",
"toasts": {
"noText": "Bitte Text eingeben",
"noModel": "Bitte ein Modell auswählen",
"generateFailed": "Generierung fehlgeschlagen"
}
},
"sound": {
"title": "Audiogenerierung",
"labels": {
"model": "Modell",
"prompt": "Prompt",
"promptPlaceholder": "Beschreiben Sie den Klang, den Sie generieren möchten...",
"duration": "Dauer (s)",
"language": "Sprache",
"vocalLanguage": "Vokalsprache",
"lyrics": "Liedtext (optional)",
"lyricsPlaceholder": "Liedtext für Vokalgenerierung",
"advanced": "Erweiterte Einstellungen",
"seed": "Seed",
"seedPlaceholder": "Zufällig"
},
"actions": {
"generate": "Generieren",
"generating": "Generieren..."
},
"empty": "Generiertes Audio erscheint hier",
"toasts": {
"noPrompt": "Bitte einen Prompt eingeben",
"noModel": "Bitte ein Modell auswählen",
"generateFailed": "Generierung fehlgeschlagen"
}
},
"talk": {
"title": "Sprechen",
"subtitle": "Sprachunterhaltung in Echtzeit",
"actions": {
"start": "Sitzung starten",
"stop": "Sitzung beenden",
"connecting": "Verbinden...",
"muted": "Stumm",
"mute": "Stummschalten",
"unmute": "Stumm aufheben"
},
"labels": {
"model": "Modell",
"voice": "Stimme",
"voicePlaceholder": "alloy",
"language": "Sprache",
"languagePlaceholder": "de",
"instructions": "Anweisungen",
"instructionsPlaceholder": "Persönlichkeit des Assistenten festlegen..."
},
"status": {
"idle": "Inaktiv",
"connecting": "Verbinden...",
"listening": "Höre zu...",
"speaking": "Spricht...",
"ended": "Sitzung beendet"
},
"toasts": {
"noModel": "Wählen Sie zuerst ein Modell",
"connectFailed": "Verbindung fehlgeschlagen: {{message}}"
}
},
"history": {
"title": "Verlauf",
"empty": "Noch kein Verlauf",
"deleteEntry": "Eintrag löschen",
"clear": "Verlauf löschen",
"clearTitle": "Gesamten Verlauf löschen",
"clearMessage": "Alle Verlaufseinträge entfernen? Diese Aktion kann nicht rückgängig gemacht werden.",
"clearConfirm": "Löschen",
"cleared": "Verlauf gelöscht"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/i18n/media.en.json
+863
View File
@@ -0,0 +1,863 @@
{
"studio": {
"tabs": {
"images": "Images",
"video": "Video",
"tts": "TTS",
"sound": "Sound",
"transform": "Transform",
"threed": "3D",
"overview": "Overview",
"diarization": "Diarization"
},
"tabState": {
"on": "(model installed)",
"off": "(no model installed)"
},
"overview": {
"title": "Studio",
"subtitle": "Images, video, 3D, speech and sound, made with the models on this machine."
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
},
"composer": {
"title": "What do you want to make?",
"lede": "Describe it and Studio suggests the type. Alt+1 to 7 picks one.",
"startTyping": "start typing",
"prompt": "Prompt",
"placeholder": {
"images": "Describe the image you want…",
"video": "Describe the shot: subject, camera move, light…",
"tts": "Type the words to speak…",
"sound": "Describe the sound or music…"
},
"fileLead": {
"threed": "3D turns one picture into an object. Start from one of your pictures below, or add a file on the next page.",
"transform": "Transform cleans up a recording or changes a voice. Start from one of your recordings below, or add a file on the next page.",
"diarization": "Diarization finds who spoke and when in a recording. Start from one of your recordings below, or add a file on the next page."
},
"starters": "Starters",
"soundsLike": "Sounds like {{type}}.",
"soundsLikeMissing": "Sounds like {{type}}, and no {{type}} model is installed yet.",
"switchTo": "Switch to {{type}}",
"seeNeeds": "See what it needs",
"model": "Model",
"noModel": "No model installed",
"size": "Size",
"count": "How many",
"countOption_one": "{{count}} image",
"countOption_other": "{{count}} images",
"startFrom": "Start from",
"sourceNone": "Start from nothing",
"sourceOnPage": "Add a file on the next page",
"fit": {
"fits": "Needs {{need}}, fits with {{amount}} GB to spare",
"spill": "Needs {{need}}, {{amount}} GB spills to RAM",
"over": "Needs {{need}}, {{amount}} GB more than this machine has"
},
"whyModel": "Install the {{type}} model first",
"whyPrompt": "Write a prompt first",
"generate": "Generate",
"openType": "Open {{type}}",
"firstRun": "Nothing is installed yet. Install the one you want first. You can add the others later, or never."
},
"starters": {
"images": [
{
"label": "Harbour at first light",
"text": "A fishing harbour at first light, pastel houses, long reflections, 35 mm film grain"
},
{
"label": "Paper-cut mountains",
"text": "Paper-cut mountains in layered teal and amber, soft haze"
},
{
"label": "Macro fern",
"text": "Macro photo of a dew-covered fern, shallow depth of field"
}
],
"video": [
{
"label": "Misty forest dolly",
"text": "Slow dolly through a misty pine forest at sunrise"
},
{
"label": "Waves on black sand",
"text": "Waves rolling onto black sand, drone shot"
}
],
"tts": [
{
"label": "Welcome line",
"text": "Welcome to the harbour tour. The first boats leave at six, and the bakery opens at five."
},
{
"label": "Release note",
"text": "Release 3.2 is out. It adds a faster scheduler and fixes the model unload bug."
}
],
"sound": [
{
"label": "Lo-fi piano",
"text": "Lo-fi piano with rain on glass"
},
{
"label": "Wind in pines",
"text": "Wind through pines, no music"
}
],
"threed": [],
"transform": [],
"diarization": []
},
"install": {
"title": "No {{type}} model installed",
"size": "is {{size}} to download.",
"sizeUnknown": "has no download size listed.",
"need": "It needs about {{need}} of memory while it runs.",
"needUnknown": "Its memory need is not listed.",
"free": "{{free}} free now.",
"nothingStarts": "Nothing starts until you install it.",
"button": "Install {{model}}",
"installing": "Installing {{model}}",
"started": "Installing {{model}}",
"failed": "The install did not start: {{message}}",
"galleryDown": "The model gallery could not be reached, so there is nothing to suggest. Check the connection, or add a model by hand.",
"noneListed": "The gallery lists no {{type}} model.",
"browse": "Browse all models"
},
"work": {
"title": "Your work",
"results_one": "{{count}} result",
"results_other": "{{count}} results",
"nResults_one": "{{count}} result",
"nResults_other": "{{count}} results",
"group": "Group related results",
"clear": "Clear history",
"clearTitle": "Clear your work history?",
"clearBody": "This removes the list of results and favourites from this browser. Files already made stay on the server.",
"clearConfirm": "Clear history",
"filters": "Filter your work",
"all": "All",
"favourites": "Favourites",
"favourite": "Favourite",
"open": "Open {{title}}",
"openProject": "Open {{title}}, {{count}} results that came from each other",
"project": "Project",
"emptyTitle": "Nothing here yet",
"emptyBody": "What you make shows up here, so you can find it again and build on it.",
"emptyFilterTitle": "No {{filter}} yet",
"emptyFilterBody": "Pick another filter, or make something.",
"fileGone": "The file is no longer on the server",
"untitled": {
"images": "Picture",
"video": "Clip",
"threed": "3D object",
"tts": "Speech",
"sound": "Sound",
"transform": "Transformed audio",
"diarization": "Recording"
}
},
"lineage": {
"back": "Your work",
"sub_one": "{{count}} result",
"sub_other": "{{count}} results that came from each other",
"subOne": "One result",
"board": "How these results were made, from the prompt on the left",
"prompt": "Prompt",
"file": "File",
"suggested": "Suggested",
"draft": "Draft",
"needsModel": "Needs a {{type}} model",
"ready": "Ready",
"dock": "Selected result",
"details": "Show details",
"take": "Run as a new take",
"branch": "Branch from here",
"cancel": "Cancel",
"openIn": "Open in {{type}}",
"newBranch": "New branch: {{step}}",
"fromResult": "from {{title}}",
"makeFrom": "Make from this result",
"targets": "What to make next",
"describe": "Describe it",
"keeps": "Keeps the original. Opens the workspace with this filled in; nothing runs until you press Generate there.",
"promptLabel": "Prompt",
"continue": "Continue from this result",
"download": "Download",
"took": "took {{seconds}} s",
"speakers_one": "{{count}} speaker",
"speakers_other": "{{count}} speakers",
"noInput": "The original input was not kept, so this cannot be run again as it was.",
"noBranch": "Nothing accepts a {{type}} as a starting point yet.",
"cannotStart": "The {{page}} page cannot start from a {{what}} yet.",
"goneTitle": "That result is no longer in this browser",
"goneBody": "It may have been cleared from the history. Go back to your work to see what is left."
},
"edges": {
"take": "take",
"animate": "animate",
"to-3d": "to 3D",
"variation": "variation",
"transform": "transform",
"diarize": "who spoke",
"soundtrack": "soundtrack",
"turntable": "turntable",
"upscale": "upscale"
},
"steps": {
"animate": {
"title": "Animate",
"why": "Turn this picture into a short clip",
"template": "Slow push-in, small natural motion"
},
"to-3d": {
"title": "Make it 3D",
"why": "Build an object you can turn around",
"plain": "3D starts from this picture. You choose the model and options on the next page."
},
"variation": {
"title": "Variation",
"why": "Another take with edits"
},
"transform": {
"title": "Transform",
"why": "Clean up this audio or change the voice",
"plain": "Transform works on this audio. You choose the model and options on the next page."
},
"diarize": {
"title": "Find who spoke",
"why": "Mark each speaker on a timeline",
"plain": "This opens the recording in Diarization. You choose the model on the next page."
},
"soundtrack": {
"title": "Add a soundtrack",
"why": "Sound that fits the shot"
},
"turntable": {
"title": "Turntable clip",
"why": "Show the object from every side"
},
"upscale": {
"title": "Upscale",
"why": "Increase the resolution of this image."
}
},
"kinds": {
"video": "video",
"mesh": "3D object"
},
"handoff": {
"aResult": "a result from Studio",
"loading": "Loading {{name}} as the starting point…",
"ready": "Starting from {{name}}.",
"missing": "The result this was made from is no longer in this browser's history, so there is no starting point. You can still add a file below.",
"error": "{{name}} could not be loaded. The server may have cleaned up the file. You can add a file below.",
"remove": "Remove"
},
"error": {
"title": "Studio could not read the installed models",
"body": "The list of models did not load, so Studio cannot say which types are ready. Your work below is not affected.",
"retry": "Try again"
},
"workspace": {
"sourcesLabel": "Starting points",
"lede": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "One picture in, one 3D object out",
"tts": "Text to speech, with a saved voice when the model clones",
"sound": "Music and sound from a description",
"transform": "Clean up a recording, or change a voice",
"diarization": "Find who spoke and when"
},
"sources": {
"startImage": "Start from an image",
"endImage": "End image",
"chosen": "Chosen file",
"audio": "Audio",
"reference": "Reference"
},
"fields": {
"prompt": "text",
"model": "model",
"size": "size",
"count": "count",
"steps": "steps",
"seed": "seed",
"negative": "negative prompt",
"duration": "duration",
"fps": "fps",
"frames": "frames",
"cfg": "CFG",
"quality": "quality",
"background": "background",
"voice": "voice",
"instructions": "delivery",
"mode": "mode",
"caption": "caption",
"lyrics": "lyrics",
"bpm": "BPM",
"key": "key",
"language": "language",
"timesignature": "time signature",
"instrumental": "instrumental",
"think": "think mode",
"params": "parameters",
"speakers": "speakers",
"detail": "detail"
},
"changedFrom": "Changed from this take:",
"edited": "edited",
"runAgain": "Run again",
"useIn": "Use in",
"rerun": "Re-run with edits",
"lineage": "Lineage",
"download": "Download",
"tools": "Result actions",
"from": "from",
"fromGone": "from a result that is no longer in this browser ({{edge}})",
"failed": {
"title": "That did not finish",
"note": "Your prompt and settings are still here.",
"retry": "Try again",
"traces": "View traces for details"
},
"strip": {
"title": "Recent results",
"allWork": "All work",
"noFavourites": "No favourites yet. Star a result to keep it here."
},
"auto": "Auto",
"sound": {
"more": "More options",
"generated": "Sound generated",
"noCaption": "Please enter caption or lyrics"
},
"transform": {
"noModel": "Please select a model",
"noAudio": "Please choose an audio file",
"done": "Audio transformed",
"inputGone": "The audio that run used is no longer on the server. Add it again to run it.",
"loadReference": "Load a reference first",
"noMic": "This browser does not expose the microphone",
"recorded": "Recorded the microphone with the reference echo. Press Transform to test echo cancellation.",
"echoFailed": "Echo test failed: {{message}}",
"recording": "recording {{seconds}} s",
"whyAudio": "Add an audio file first",
"waiting": "Your audio, waiting to be transformed",
"paramCount_one": "{{count}} parameter",
"paramCount_other": "{{count}} parameters"
},
"threed": {
"whyImage": "Add a picture first"
},
"choose": "Choose",
"replace": "Replace",
"dropHint": "or drop it here",
"diarization": {
"whyFile": "Choose a recording first",
"hint": "Audio or video, or drop it here",
"timeline": "Timeline of {{count}} speakers over {{length}}",
"segmentCount_one": "{{count}} segment",
"segmentCount_other": "{{count}} segments",
"talk": "{{seconds}} s of talk, {{percent}}%",
"notKept": "The recording and the transcript are not kept, so only the summary of this run is shown. Run it again with the file to see the timeline.",
"empty": "Choose a recording to see who spoke and when"
}
},
"upscale": {
"model": "Upscaling model",
"scale": "Scale",
"source": "Source image",
"loading": "Upscaling…",
"failed": "Upscale failed: {{message}}",
"noResult": "No output image was returned."
}
},
"image": {
"title": "Image Generation",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Describe the image you want to generate...",
"negativePrompt": "Negative Prompt",
"negativePromptPlaceholder": "What to avoid...",
"size": "Size",
"count": "Count (1-4)",
"advanced": "Advanced Settings",
"imageInputs": "Image Inputs",
"steps": "Steps",
"stepsPlaceholder": "20",
"seed": "Seed",
"seedPlaceholder": "Random",
"sourceImage": "Source Image (img2img)",
"refImages": "Reference Images",
"refImagesAdded_one": "{{count}} image added",
"refImagesAdded_other": "{{count}} images added"
},
"actions": {
"view": "View",
"generate": "Generate",
"generating": "Generating..."
},
"empty": "Generated images will appear here",
"toasts": {
"noPrompt": "Please enter a prompt",
"noModel": "Please select a model",
"noResults": "No images generated"
}
},
"video": {
"title": "Video Generation",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Describe the video you want to generate...",
"duration": "Duration (seconds)",
"fps": "FPS",
"size": "Size",
"advanced": "Advanced Settings",
"seed": "Seed",
"seedPlaceholder": "Random",
"frames": "Frames",
"referenceMedia": "Reference media",
"startImage": "Start image",
"endImage": "End image",
"avatarAudio": "Avatar audio"
},
"actions": {
"generate": "Generate",
"generating": "Generating..."
},
"empty": "Generated video will appear here",
"toasts": {
"noPrompt": "Please enter a prompt",
"noModel": "Please select a model",
"noResults": "No video generated"
}
},
"threed": {
"title": "3D Generation",
"animation": {
"skeletonHint": "Generates an animated skeleton, without a mesh or skin.",
"preview": "Animated skeleton preview",
"pause": "Pause",
"play": "Play",
"reset": "Reset view",
"timeline": "Timeline",
"zoom": "Zoom",
"inputTooLong": "Input exceeds the {{limit}} byte limit.",
"fileTooLarge": "The selected file is too large.",
"readFailed": "Could not read the selected file."
},
"labels": {
"model": "Model",
"image": "Input image",
"quality": "Quality",
"quality_auto": "Auto (best available)",
"quality_coarse": "Coarse preview (fast)",
"quality_512": "512³ fine",
"quality_1024": "1024³ cascade (slow)",
"background": "Background",
"background_auto": "Auto-remove solid background",
"background_keep": "Keep original",
"background_black": "Remove black",
"background_white": "Remove white",
"advanced": "Advanced Settings",
"steps": "Shape steps",
"textureSteps": "Material steps",
"guidance": "Guidance",
"seed": "Seed",
"seedPlaceholder": "Random"
},
"actions": {
"generate": "Generate",
"generating": "Generating...",
"remesh": "Apply remeshing",
"remeshing": "Applying remeshing...",
"showOriginal": "Show original",
"download": "Download GLB"
},
"remesh": {
"title": "Watertight print remesh",
"detail": "Remesh detail",
"coarser": "Coarser · faster",
"finer": "Finer · slower",
"hint": "Detail controls the smallest preserved features. The enclosing offset follows it automatically.",
"ready": "Previewing the watertight remeshed model. This is the version that will be downloaded."
},
"viewer": {
"wireframe": "Wireframe",
"autoRotate": "Auto-rotate",
"noWebgl": "WebGL2 is not available in this browser — download the GLB to view it elsewhere.",
"contextLost": "The 3D view lost the GPU context — reload the page to restore it.",
"stats": "{{verts}} vertices · {{tris}} triangles",
"hint": "drag: rotate · pinch/wheel: zoom · two-finger/right-drag: pan · double-click: reset"
},
"empty": "Generated 3D model will appear here",
"toasts": {
"noImage": "Please provide an input image",
"noModel": "Please select a model",
"noResults": "No model generated"
}
},
"tts": {
"title": "Text to Speech",
"labels": {
"model": "Model",
"voice": "Voice",
"voicePlaceholder": "Optional speaker or voice ID",
"input": "Text",
"inputPlaceholder": "Enter text to synthesize...",
"instructions": "Instructions",
"instructionsPlaceholder": "For example: Speak slowly and warmly",
"instructionsHint": "Optional delivery guidance. Support depends on the backend."
},
"actions": {
"generate": "Generate",
"generating": "Generating..."
},
"empty": "Generated audio will appear here",
"toasts": {
"noText": "Please enter text",
"noModel": "Please select a model",
"generated": "Speech generated",
"generateFailed": "Generation failed"
},
"voiceLibrary": {
"cloningReady": "Voice cloning",
"loading": "Loading saved personalities…",
"modelDefault": "Use model default",
"empty": "No saved personalities yet.",
"create": "Create one",
"manage": "Manage Personality Library",
"namedVoiceHint": "This model does not accept reference-audio profiles. You can still enter a named speaker supported by its backend."
}
},
"voiceLibrary": {
"title": "Speech voices",
"subtitle": "A recording and its exact transcript. Text-to-speech models that accept reference audio can speak with it.",
"loadFailed": "Speech voices could not be loaded",
"loading": "Loading saved personalities…",
"listLabel": "Saved voice personalities",
"status": {
"ready": "Ready"
},
"summary": {
"label": "Voice library status",
"profiles": "saved personalities",
"modelsReady_one": "{{count}} compatible model ready",
"modelsReady_other": "{{count}} compatible models ready",
"noModels": "No compatible model installed"
},
"search": {
"label": "Search voice profiles",
"placeholder": "Search voices, languages, or transcripts"
},
"filters": {
"language": "Filter by language",
"allLanguages": "All languages"
},
"empty": {
"title": "No speech voices yet",
"body": "Record or upload a consented reference clip, add its exact transcript, and reuse the voice across compatible text-to-speech models."
},
"noResults": {
"title": "No voices match these filters",
"body": "Try a different search or show every language."
},
"detail": {
"eyebrow": "Reference recording and exact transcript",
"close": "Close",
"emptyTitle": "Select a voice to inspect it",
"emptyBody": "Reference audio, transcript, compatibility, and consent details will appear here.",
"referenceAudio": "Reference audio",
"transcript": "Reference transcript"
},
"metadata": {
"language": "Language",
"languageUnknown": "Not specified",
"duration": "Duration",
"sampleRate": "Sample rate",
"created": "Created"
},
"consent": {
"label": "Permission",
"confirmed": "Consent confirmed",
"confirmedAt": "Confirmed {{date}}"
},
"modelSetup": {
"title": "Install a voice-cloning model",
"body": "This voice is ready, but it needs a model that accepts reference audio. Choose one verified by this server.",
"loading": "Finding compatible models in your galleries…",
"backendUnknown": "Backend selected during install",
"install": "Install",
"installing": "Installing…",
"loadFailed": "Compatible recommendations could not be loaded.",
"noneAvailable": "No compatible uninstalled model is available in the configured galleries.",
"capabilityNote": "Recommendations come from model capability metadata, not a browser-side model list."
},
"api": {
"title": "API usage and compatible models",
"summary": "Use this voice outside the browser or check model support",
"body": "Send the profile’s stable voice URI with each speech request. LocalAI resolves the private reference audio and transcript at generation time; model YAML is not modified.",
"compatibleModels": "Compatible installed models",
"noInstalledModels": "None yet. Install one of the server-verified models above before sending this request.",
"curlExample": "cURL example",
"copy": "Copy request",
"endpointNote": "The same model, input, and voice fields also work with POST /tts."
},
"actions": {
"create": "Add a speech voice",
"createFirst": "Add your first speech voice",
"retry": "Try again",
"clearFilters": "Clear filters",
"useInTTS": "Use in Text to Speech",
"delete": "Delete voice",
"installModel": "Install a compatible model",
"browseModels": "Browse all models"
},
"deleteDialog": {
"title": "Delete this voice?",
"message": "“{{name}}” and its reference audio will be permanently removed. Existing model YAML is not changed.",
"deleting": "Deleting…"
},
"toasts": {
"deleted": "Deleted {{name}}",
"installStarted": "Installing {{name}}. Progress is available in Operations.",
"installFailed": "Could not start installation: {{message}}",
"apiCopied": "API request copied",
"copyFailed": "Could not copy the API request"
}
},
"voiceCreate": {
"title": "Add a speech voice",
"subtitle": "A speech voice is one or more clean reference clips and their exact transcripts. Text-to-speech models that accept reference audio can speak with it.",
"references": {
"add": "Add another reference",
"additional": "Reference {{number}}",
"remove": "Remove"
},
"sections": {
"reference": {
"title": "Reference audio",
"body": "Record in the browser or upload an existing clip. Speech should be clean, natural, and from one speaker."
},
"details": {
"title": "Identity and transcript",
"body": "Give admins a recognizable name and transcribe the clip exactly, including fillers and punctuation."
},
"consent": {
"title": "Authorization",
"body": "Voice cloning can be sensitive. Confirm permission before this reference is saved."
}
},
"audio": {
"label": "Voice reference",
"normalizing": "Preparing a secure PCM WAV…",
"preview": "Normalized reference",
"tooLarge": "This file is larger than 50 MiB.",
"durationError": "Reference audio must be between 1 second and 2 minutes.",
"decodeError": "This browser could not decode the selected audio file.",
"qualityReady": "Recommended length",
"qualityHint": "6–30 seconds usually produces a stronger clone"
},
"fields": {
"name": "Voice name",
"namePlaceholder": "e.g. Documentary narrator",
"language": "Language (optional)",
"languagePlaceholder": "e.g. en-US",
"description": "Description (optional)",
"descriptionPlaceholder": "Tone, source, or intended use",
"transcript": "Exact transcript",
"transcriptPlaceholder": "Type every word spoken in the reference clip…",
"transcriptHint": "Match the recording exactly. Do not rewrite grammar or remove pauses and fillers."
},
"consent": {
"title": "I confirm that this voice may be cloned",
"body": "I have the speaker’s permission or another lawful basis to store and use this recording for synthesized speech."
},
"readiness": {
"title": "Profile readiness",
"body": "A voice becomes available in Text to Speech when every required item is complete.",
"audio": "Valid PCM-WAV reference",
"quality": "6–30 second quality window",
"name": "Recognizable voice name",
"transcript": "Exact reference transcript",
"consent": "Authorization confirmed"
},
"privacy": {
"title": "Private by default. ",
"body": "Reference audio is stored in LocalAI’s data directory with restricted file permissions and is never exposed as a model-folder path."
},
"actions": {
"back": "Back to library",
"cancel": "Cancel",
"save": "Save voice",
"saving": "Saving…"
},
"toasts": {
"created": "{{name}} is ready to use"
}
},
"sound": {
"title": "Sound Generation",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Describe the sound you want to generate...",
"instrumental": "Instrumental",
"bpm": "BPM",
"duration": "Duration (seconds)",
"language": "Language",
"timesignature": "Time Signature",
"keyscale": "Key/Scale",
"vocalLanguage": "Vocal language",
"vocalLanguagePlaceholder": "e.g. English",
"caption": "Caption",
"lyrics": "Lyrics",
"lyricsPlaceholder": "Lyrics for vocal generation",
"simple": "Simple",
"advanced": "Advanced",
"seed": "Seed",
"seedPlaceholder": "Random",
"thinkMode": "Think mode"
},
"actions": {
"generate": "Generate",
"generating": "Generating..."
},
"empty": "Generated audio will appear here",
"toasts": {
"noPrompt": "Please enter a prompt",
"noModel": "Please select a model",
"generateFailed": "Generation failed"
},
"expansion": {
"label": "Prompt expansion model",
"placeholder": "Choose a model or backend default",
"backendDefault": "Backend default",
"choose": "Choose a chat model or Backend default to continue.",
"loading": "Loading installed chat models…",
"error": "Cannot load chat models. Choose Backend default or return to this tab to retry.",
"noChat": "No installed chat models. Choose Backend default to use the audio backend.",
"help": "A chat model expands your description into style and lyrics. Backend default uses the audio backend’s existing prompt behavior."
}
},
"audioTransform": {
"title": "Audio Transform",
"labels": {
"model": "Model",
"audio": "Audio (required)",
"reference": "Reference (optional)",
"referenceHelp": "Loopback / far-end signal for echo cancellation, target speaker for voice conversion. Leave empty for unconditional transform.",
"advancedParameters": "Advanced parameters",
"advancedParametersHelp": "backend-specific (one key=value per line, e.g. noise_gate=true)",
"advancedParametersPlaceholder": "# Optional. For LocalVQE:\n# noise_gate=true\n# noise_gate_threshold_dbfs=-50"
},
"input": {
"upload": "Upload",
"uploadDescription": "Drop a file here or",
"uploadBrowse": "browse",
"record": "Record",
"startRecording": "Start recording",
"stop": "Stop",
"encoding": "Encoding...",
"clear": "Clear",
"microphoneUnavailable": "Microphone capture is unavailable in this browser.",
"echoNotice": "Browsers often apply their own WebRTC echo cancellation and noise suppression by default. This usually results in worse performance than running LocalVQE on the raw audio.",
"echoTest": "Echo test (record mic while playing reference)",
"stopEchoTest": "Stop echo test"
},
"result": {
"audio": "Audio",
"reference": "Reference",
"inputSpectrum": "Input spectrum",
"outputSpectrum": "Output spectrum",
"outputSpectrumHint": "Transform to compare attenuation",
"output": "Output"
},
"actions": {
"transform": "Transform",
"processing": "Processing..."
},
"empty": "Choose an audio file (and optional reference) to transform"
},
"talk": {
"title": "Talk",
"subtitle": "Realtime voice conversation",
"actions": {
"start": "Start session",
"stop": "End session",
"connecting": "Connecting...",
"muted": "Muted",
"mute": "Mute",
"unmute": "Unmute"
},
"labels": {
"model": "Model",
"voice": "Voice",
"voicePlaceholder": "alloy",
"language": "Language",
"languagePlaceholder": "en",
"instructions": "Instructions",
"instructionsPlaceholder": "Set the assistant's persona..."
},
"status": {
"idle": "Idle",
"connecting": "Connecting...",
"listening": "Listening...",
"speaking": "Speaking...",
"ended": "Session ended"
},
"toasts": {
"noModel": "Select a model first",
"connectFailed": "Failed to connect: {{message}}"
}
},
"history": {
"title": "History",
"empty": "No history yet",
"deleteEntry": "Delete entry",
"clear": "Clear history",
"clearTitle": "Clear all history",
"clearMessage": "Remove all history entries? This cannot be undone.",
"clearConfirm": "Clear",
"cleared": "History cleared"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
},
"diarization": {
"title": "Speaker diarization",
"subtitle": "Upload a recording to see who spoke when. Preview clean speech before remembering a voice.",
"model": "Model",
"recording": "Recording",
"optIn": "Prepare speakers to remember",
"warning": "Remembered voices are shared globally on this server and are lost when it restarts. Nothing is remembered automatically.",
"manage": "Manage remembered voices",
"running": "Diarizing…",
"run": "Diarize",
"unsupported": "Choose a profile-capable model with a speaker encoder, or turn off “Prepare speakers to remember” to run normal diarization.",
"previewError": "Could not play this recording. Check that your browser supports its audio format.",
"speakers": "Speakers",
"segments": "Segments",
"duration": "Clean speech: {{seconds}} seconds",
"preview": "Preview {{number}}",
"insufficient": "Not enough clean speech or no usable profile. Try a recording with longer, non-overlapping speech.",
"nameAndRemember": "Name and remember",
"name": "Name",
"saving": "Saving…",
"remember": "Remember",
"cancel": "Cancel",
"stop": "Stop preview",
"missingProfiles": "The backend did not return the requested speaker profiles."
}
}
addons/studio/ui/src/i18n/media.es.json
+195
View File
@@ -0,0 +1,195 @@
{
"studio": {
"tabs": {
"images": "Imágenes",
"video": "Video",
"tts": "TTS",
"sound": "Sonido",
"transform": "Transform",
"overview": "Overview"
},
"overview": {
"eyebrow": "{{ready}} of {{total}} modalities ready",
"title": "Studio",
"subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
"canMake": "What you can make",
"running": "Running now",
"recent": "Recent outputs",
"noModel": "No model installed",
"install": "Install a model",
"ready": "Ready",
"seconds": "{{seconds}}s",
"describe": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "Image to mesh reconstruction",
"tts": "Text to speech using your voice library",
"sound": "Music and sound effects from a prompt",
"transform": "Separation, enhancement and voice conversion"
}
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
}
},
"image": {
"title": "Generación de imágenes",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Describe la imagen que quieres generar...",
"negativePrompt": "Prompt negativo",
"negativePromptPlaceholder": "Qué evitar...",
"size": "Tamaño",
"count": "Cantidad (1-4)",
"advanced": "Configuración avanzada",
"imageInputs": "Entradas de imagen",
"steps": "Pasos",
"stepsPlaceholder": "20",
"seed": "Seed",
"seedPlaceholder": "Aleatorio",
"sourceImage": "Imagen fuente (img2img)",
"refImages": "Imágenes de referencia",
"refImagesAdded_one": "{{count}} imagen añadida",
"refImagesAdded_other": "{{count}} imágenes añadidas"
},
"actions": {
"view": "View",
"generate": "Generar",
"generating": "Generando..."
},
"empty": "Las imágenes generadas aparecerán aquí",
"toasts": {
"noPrompt": "Por favor introduce un prompt",
"noModel": "Por favor selecciona un modelo",
"noResults": "No se generaron imágenes"
}
},
"video": {
"title": "Generación de video",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Describe el video que quieres generar...",
"duration": "Duración (s)",
"fps": "FPS",
"size": "Tamaño",
"advanced": "Configuración avanzada",
"seed": "Seed",
"seedPlaceholder": "Aleatorio",
"frames": "Fotogramas",
"referenceMedia": "Medios de referencia",
"startImage": "Imagen inicial",
"endImage": "Imagen final",
"avatarAudio": "Audio del avatar"
},
"actions": {
"generate": "Generar video",
"generating": "Generando..."
},
"empty": "Los videos generados aparecerán aquí",
"toasts": {
"noPrompt": "Por favor introduce un prompt",
"noModel": "Por favor selecciona un modelo",
"noResults": "No se generaron videos"
}
},
"tts": {
"title": "Texto a voz",
"labels": {
"model": "Modelo",
"voice": "Voz",
"voicePlaceholder": "ID de voz opcional",
"input": "Texto",
"inputPlaceholder": "Introduce el texto a sintetizar...",
"instructions": "Instrucciones",
"instructionsPlaceholder": "Por ejemplo: Habla despacio y con calidez",
"instructionsHint": "Indicaciones opcionales sobre la locución. La compatibilidad depende del backend."
},
"actions": {
"generate": "Generar audio",
"generating": "Generando..."
},
"empty": "El audio generado aparecerá aquí",
"toasts": {
"noText": "Por favor introduce texto",
"noModel": "Por favor selecciona un modelo",
"generateFailed": "Generación fallida"
}
},
"sound": {
"title": "Generación de sonido",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Describe el sonido que quieres generar...",
"duration": "Duración (s)",
"language": "Idioma",
"vocalLanguage": "Idioma vocal",
"lyrics": "Letras (opcional)",
"lyricsPlaceholder": "Letras para generación vocal",
"advanced": "Configuración avanzada",
"seed": "Seed",
"seedPlaceholder": "Aleatorio"
},
"actions": {
"generate": "Generar",
"generating": "Generando..."
},
"empty": "El audio generado aparecerá aquí",
"toasts": {
"noPrompt": "Por favor introduce un prompt",
"noModel": "Por favor selecciona un modelo",
"generateFailed": "Generación fallida"
}
},
"talk": {
"title": "Hablar",
"subtitle": "Conversación de voz en tiempo real",
"actions": {
"start": "Iniciar sesión",
"stop": "Terminar sesión",
"connecting": "Conectando...",
"muted": "Silenciado",
"mute": "Silenciar",
"unmute": "Activar sonido"
},
"labels": {
"model": "Modelo",
"voice": "Voz",
"voicePlaceholder": "alloy",
"language": "Idioma",
"languagePlaceholder": "es",
"instructions": "Instrucciones",
"instructionsPlaceholder": "Define la personalidad del asistente..."
},
"status": {
"idle": "Inactivo",
"connecting": "Conectando...",
"listening": "Escuchando...",
"speaking": "Hablando...",
"ended": "Sesión finalizada"
},
"toasts": {
"noModel": "Selecciona un modelo primero",
"connectFailed": "Error al conectar: {{message}}"
}
},
"history": {
"title": "Historial",
"empty": "Sin historial aún",
"deleteEntry": "Eliminar entrada",
"clear": "Borrar historial",
"clearTitle": "Borrar todo el historial",
"clearMessage": "¿Eliminar todas las entradas del historial? Esto no se puede deshacer.",
"clearConfirm": "Borrar",
"cleared": "Historial borrado"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/i18n/media.id.json
+242
View File
@@ -0,0 +1,242 @@
{
"studio": {
"tabs": {
"images": "Gambar",
"video": "Video",
"tts": "TTS",
"sound": "Suara",
"transform": "Transformasi",
"overview": "Ikhtisar"
},
"overview": {
"eyebrow": "{{ready}} dari {{total}} modalitas siap",
"title": "Studio",
"subtitle": "Hasilkan gambar, video, 3D dan suara dengan model pada mesin ini.",
"canMake": "Apa yang dapat Anda buat",
"running": "Sedang berjalan",
"recent": "Output terbaru",
"noModel": "Tidak ada model yang terinstal",
"install": "Instal model",
"ready": "Siap",
"seconds": "{{seconds}}s",
"describe": {
"images": "Teks ke gambar, gambar ke gambar, gambar referensi",
"video": "Teks ke video dan gambar ke video",
"threed": "Rekonstruksi mesh dari gambar",
"tts": "Teks ke suara menggunakan pustaka suara Anda",
"sound": "Musik dan efek suara dari prompt",
"transform": "Pemisahan, peningkatan dan konversi suara"
}
},
"groups": {
"create": "Buat",
"voice": "Suara",
"transform": "Transformasi"
}
},
"image": {
"title": "Pembuatan Gambar",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Deskripsikan gambar yang ingin Anda buat...",
"negativePrompt": "Prompt Negatif",
"negativePromptPlaceholder": "Apa yang harus dihindari...",
"size": "Ukuran",
"count": "Jumlah (1-4)",
"advanced": "Pengaturan Tingkat Lanjutan",
"imageInputs": "Input Gambar",
"steps": "Steps",
"stepsPlaceholder": "20",
"seed": "Seed",
"seedPlaceholder": "Acak",
"sourceImage": "Gambar Sumber (img2img)",
"refImages": "Gambar Referensi",
"refImagesAdded_one": "{{count}} gambar ditambahkan",
"refImagesAdded_other": "{{count}} gambar ditambahkan"
},
"actions": {
"view": "Lihat",
"generate": "Hasilkan",
"generating": "Menghasilkan..."
},
"empty": "Gambar yang dihasilkan akan muncul di sini",
"toasts": {
"noPrompt": "Silahkan masukkan prompt",
"noModel": "Silahkan pilih model",
"noResults": "Tidak ada gambar yang dihasilkan"
}
},
"video": {
"title": "Pembuatan Video",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Deskripsikan video yang ingin Anda buat...",
"duration": "Durasi (detik)",
"fps": "FPS",
"size": "Ukuran",
"advanced": "Pengaturan Tingkat Lanjutan",
"seed": "Seed",
"seedPlaceholder": "Acak",
"frames": "Frame",
"referenceMedia": "Media referensi",
"startImage": "Gambar awal",
"endImage": "Gambar akhir",
"avatarAudio": "Audio avatar"
},
"actions": {
"generate": "Hasilkan",
"generating": "Menghasilkan..."
},
"empty": "Video yang dihasilkan akan muncul di sini",
"toasts": {
"noPrompt": "Silahkan masukkan prompt",
"noModel": "Silahkan pilih model",
"noResults": "Tidak ada video yang dihasilkan"
}
},
"tts": {
"title": "Teks ke Suara",
"labels": {
"model": "Model",
"voice": "Suara",
"voicePlaceholder": "ID suara opsional",
"input": "Teks",
"inputPlaceholder": "Masukkan teks untuk disintesis...",
"instructions": "Instruksi",
"instructionsPlaceholder": "Contoh: Ucapkan perlahan dan hangat",
"instructionsHint": "Panduan penyampaian opsional. Dukungan bergantung pada backend."
},
"actions": {
"generate": "Hasilkan",
"generating": "Menghasilkan..."
},
"empty": "Audio yang dihasilkan akan muncul di sini",
"toasts": {
"noText": "Silahkan masukkan teks",
"noModel": "Silahkan pilih model",
"generateFailed": "Gagal menghasilkan"
}
},
"sound": {
"title": "Pembuatan Suara",
"labels": {
"model": "Model",
"prompt": "Prompt",
"promptPlaceholder": "Deskripsikan suara yang ingin Anda buat...",
"instrumental": "Instrumental",
"bpm": "Tempo",
"duration": "Durasi (detik)",
"language": "Bahasa",
"timesignature": "Tanda birama",
"keyscale": "Tangga nada",
"vocalLanguage": "Bahasa vokal",
"vocalLanguagePlaceholder": "cth. Bahasa Indonesia",
"caption": "Takarir",
"lyrics": "Lirik",
"lyricsPlaceholder": "Lirik untuk menghasilkan vokal",
"simple": "Sederhana",
"advanced": "Tingkat Lanjutan",
"seed": "Seed",
"seedPlaceholder": "Acak",
"thinkMode": "Mode penalaran"
},
"actions": {
"generate": "Hasilkan",
"generating": "Menghasilkan..."
},
"empty": "Suara yang dihasilkan akan muncul di sini",
"toasts": {
"noPrompt": "Silahkan masukkan prompt",
"noModel": "Silahkan pilih model",
"generateFailed": "Gagal menghasilkan"
}
},
"audioTransform": {
"title": "Transformasi Audio",
"labels": {
"model": "Model",
"audio": "Audio (wajib)",
"reference": "Referensi (opsional)",
"referenceHelp": "Sinyal loopback / ujung jauh untuk pembatalan gema, atau pembicara target untuk konversi suara. Kosongkan untuk transformasi tanpa syarat.",
"advancedParameters": "Parameter tingkat lanjut",
"advancedParametersHelp": "Spesifik backend (satu key=value per baris, contoh: noise_gate=true)",
"advancedParametersPlaceholder": "# Opsional. Untuk LocalVQE:\n# noise_gate=true\n# noise_gate_threshold_dbfs=-50"
},
"input": {
"upload": "Unggah",
"uploadDescription": "Tarik file di sini atau",
"uploadBrowse": "jelajahi",
"record": "Rekam",
"startRecording": "Mulai merekam",
"stop": "Selesai",
"encoding": "Pengenkodean...",
"clear": "Hapus",
"microphoneUnavailable": "Perekaman mikrofon tidak tersedia di peramban ini.",
"echoNotice": "Peramban sering kali menerapkan pembatalan gema dan penekanan derau WebRTC secara bawaan. Hal ini biasanya menghasilkan performa yang lebih buruk dibandingkan menjalankan LocalVQE pada audio mentah.",
"echoTest": "Tes gema (rekam mik saat memutar referensi)",
"stopEchoTest": "Hentikan tes gema"
},
"result": {
"audio": "Audio",
"reference": "Referensi",
"inputSpectrum": "Input spektrum",
"outputSpectrum": "Output spektrum",
"outputSpectrumHint": "Transformasi untuk membandingkan atenuasi",
"output": "Output"
},
"actions": {
"transform": "Transformasi",
"processing": "Memproses..."
},
"empty": "Pilih file audio (dan referensi opsional) untuk ditransformasikan"
},
"talk": {
"title": "Percakapan",
"subtitle": "Percakapan suara realtime",
"actions": {
"start": "Mulai sesi",
"stop": "Akhiri sesi",
"connecting": "Menghubungkan...",
"muted": "Disenyapkan",
"mute": "Senyapkan",
"unmute": "Aktifkan Suara"
},
"labels": {
"model": "Model",
"voice": "Suara",
"voicePlaceholder": "alloy",
"language": "Bahasa",
"languagePlaceholder": "en",
"instructions": "Instruksi",
"instructionsPlaceholder": "Atur persona asisten..."
},
"status": {
"idle": "Idle",
"connecting": "Menghubungkan...",
"listening": "Mendengarkan...",
"speaking": "Berbicara...",
"ended": "Sesi berakhir"
},
"toasts": {
"noModel": "Silahkan pilih model terlebih dahulu",
"connectFailed": "Gagal terhubung: {{message}}"
}
},
"history": {
"title": "Riwayat",
"empty": "Tidak ada riwayat",
"deleteEntry": "Hapus entri",
"clear": "Hapus riwayat",
"clearTitle": "Hapus semua riwayat",
"clearMessage": "Hapus semua entri riwayat? Tindakan ini tidak dapat dibatalkan.",
"clearConfirm": "Hapus",
"cleared": "Riwayat dihapus"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/i18n/media.it.json
+195
View File
@@ -0,0 +1,195 @@
{
"studio": {
"tabs": {
"images": "Immagini",
"video": "Video",
"tts": "TTS",
"sound": "Audio",
"transform": "Transform",
"overview": "Overview"
},
"overview": {
"eyebrow": "{{ready}} of {{total}} modalities ready",
"title": "Studio",
"subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
"canMake": "What you can make",
"running": "Running now",
"recent": "Recent outputs",
"noModel": "No model installed",
"install": "Install a model",
"ready": "Ready",
"seconds": "{{seconds}}s",
"describe": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "Image to mesh reconstruction",
"tts": "Text to speech using your voice library",
"sound": "Music and sound effects from a prompt",
"transform": "Separation, enhancement and voice conversion"
}
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
}
},
"image": {
"title": "Generazione immagini",
"labels": {
"model": "Modello",
"prompt": "Prompt",
"promptPlaceholder": "Descrivi l'immagine che vuoi generare...",
"negativePrompt": "Prompt negativo",
"negativePromptPlaceholder": "Cosa evitare...",
"size": "Dimensione",
"count": "Quantità (1-4)",
"advanced": "Impostazioni avanzate",
"imageInputs": "Input immagini",
"steps": "Passi",
"stepsPlaceholder": "20",
"seed": "Seed",
"seedPlaceholder": "Casuale",
"sourceImage": "Immagine sorgente (img2img)",
"refImages": "Immagini di riferimento",
"refImagesAdded_one": "{{count}} immagine aggiunta",
"refImagesAdded_other": "{{count}} immagini aggiunte"
},
"actions": {
"view": "Visualizza",
"generate": "Genera",
"generating": "Generazione..."
},
"empty": "Le immagini generate appariranno qui",
"toasts": {
"noPrompt": "Inserisci un prompt",
"noModel": "Seleziona un modello",
"noResults": "Nessuna immagine generata"
}
},
"video": {
"title": "Generazione video",
"labels": {
"model": "Modello",
"prompt": "Prompt",
"promptPlaceholder": "Descrivi il video che vuoi generare...",
"duration": "Durata (s)",
"fps": "FPS",
"size": "Dimensione",
"advanced": "Impostazioni avanzate",
"seed": "Seed",
"seedPlaceholder": "Casuale",
"frames": "Fotogrammi",
"referenceMedia": "Media di riferimento",
"startImage": "Immagine iniziale",
"endImage": "Immagine finale",
"avatarAudio": "Audio avatar"
},
"actions": {
"generate": "Genera video",
"generating": "Generazione..."
},
"empty": "I video generati appariranno qui",
"toasts": {
"noPrompt": "Inserisci un prompt",
"noModel": "Seleziona un modello",
"noResults": "Nessun video generato"
}
},
"tts": {
"title": "Text to Speech",
"labels": {
"model": "Modello",
"voice": "Voce",
"voicePlaceholder": "ID voce opzionale",
"input": "Testo",
"inputPlaceholder": "Inserisci il testo da convertire in voce...",
"instructions": "Istruzioni",
"instructionsPlaceholder": "Ad esempio: Parla lentamente e con calore",
"instructionsHint": "Indicazioni facoltative sullo stile. Il supporto dipende dal backend."
},
"actions": {
"generate": "Genera audio",
"generating": "Generazione..."
},
"empty": "L'audio generato apparirà qui",
"toasts": {
"noText": "Inserisci del testo",
"noModel": "Seleziona un modello",
"generateFailed": "Generazione fallita"
}
},
"sound": {
"title": "Generazione audio",
"labels": {
"model": "Modello",
"prompt": "Prompt",
"promptPlaceholder": "Descrivi il suono che vuoi generare...",
"duration": "Durata (s)",
"language": "Lingua",
"vocalLanguage": "Lingua vocale",
"lyrics": "Testi (opzionale)",
"lyricsPlaceholder": "Testi per la generazione vocale",
"advanced": "Impostazioni avanzate",
"seed": "Seed",
"seedPlaceholder": "Casuale"
},
"actions": {
"generate": "Genera",
"generating": "Generazione..."
},
"empty": "L'audio generato apparirà qui",
"toasts": {
"noPrompt": "Inserisci un prompt",
"noModel": "Seleziona un modello",
"generateFailed": "Generazione fallita"
}
},
"talk": {
"title": "Conversazione",
"subtitle": "Conversazione vocale in tempo reale",
"actions": {
"start": "Avvia sessione",
"stop": "Termina sessione",
"connecting": "Connessione...",
"muted": "Muto",
"mute": "Disattiva microfono",
"unmute": "Riattiva microfono"
},
"labels": {
"model": "Modello",
"voice": "Voce",
"voicePlaceholder": "alloy",
"language": "Lingua",
"languagePlaceholder": "it",
"instructions": "Istruzioni",
"instructionsPlaceholder": "Imposta la personalità dell'assistente..."
},
"status": {
"idle": "Inattivo",
"connecting": "Connessione...",
"listening": "In ascolto...",
"speaking": "Sta parlando...",
"ended": "Sessione terminata"
},
"toasts": {
"noModel": "Seleziona prima un modello",
"connectFailed": "Connessione fallita: {{message}}"
}
},
"history": {
"title": "Cronologia",
"empty": "Nessuna cronologia",
"deleteEntry": "Elimina voce",
"clear": "Cancella cronologia",
"clearTitle": "Cancella tutta la cronologia",
"clearMessage": "Rimuovere tutte le voci della cronologia? Questa azione non può essere annullata.",
"clearConfirm": "Cancella",
"cleared": "Cronologia cancellata"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/i18n/media.ko.json
+195
View File
@@ -0,0 +1,195 @@
{
"studio": {
"tabs": {
"images": "이미지",
"video": "비디오",
"tts": "TTS",
"sound": "사운드",
"transform": "Transform",
"overview": "Overview"
},
"overview": {
"eyebrow": "{{ready}} of {{total}} modalities ready",
"title": "Studio",
"subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
"canMake": "What you can make",
"running": "Running now",
"recent": "Recent outputs",
"noModel": "No model installed",
"install": "Install a model",
"ready": "Ready",
"seconds": "{{seconds}}s",
"describe": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "Image to mesh reconstruction",
"tts": "Text to speech using your voice library",
"sound": "Music and sound effects from a prompt",
"transform": "Separation, enhancement and voice conversion"
}
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
}
},
"image": {
"title": "이미지 생성",
"labels": {
"model": "모델",
"prompt": "프롬프트",
"promptPlaceholder": "생성할 이미지를 설명하세요...",
"negativePrompt": "네거티브 프롬프트",
"negativePromptPlaceholder": "피할 요소...",
"size": "크기",
"count": "개수 (1-4)",
"advanced": "고급 설정",
"imageInputs": "이미지 입력",
"steps": "스텝",
"stepsPlaceholder": "20",
"seed": "시드",
"seedPlaceholder": "랜덤",
"sourceImage": "원본 이미지 (img2img)",
"refImages": "참조 이미지",
"refImagesAdded_one": "이미지 {{count}}개 추가됨",
"refImagesAdded_other": "이미지 {{count}}개 추가됨"
},
"actions": {
"view": "View",
"generate": "생성",
"generating": "생성 중..."
},
"empty": "생성된 이미지가 여기에 표시됩니다",
"toasts": {
"noPrompt": "프롬프트를 입력해 주세요",
"noModel": "모델을 선택해 주세요",
"noResults": "생성된 이미지가 없습니다"
}
},
"video": {
"title": "비디오 생성",
"labels": {
"model": "모델",
"prompt": "프롬프트",
"promptPlaceholder": "생성할 비디오를 설명하세요...",
"duration": "길이 (초)",
"fps": "FPS",
"size": "크기",
"advanced": "고급 설정",
"seed": "시드",
"seedPlaceholder": "랜덤",
"frames": "프레임",
"referenceMedia": "참조 미디어",
"startImage": "시작 이미지",
"endImage": "종료 이미지",
"avatarAudio": "아바타 오디오"
},
"actions": {
"generate": "생성",
"generating": "생성 중..."
},
"empty": "생성된 비디오가 여기에 표시됩니다",
"toasts": {
"noPrompt": "프롬프트를 입력해 주세요",
"noModel": "모델을 선택해 주세요",
"noResults": "생성된 비디오가 없습니다"
}
},
"tts": {
"title": "텍스트 음성 변환",
"labels": {
"model": "모델",
"voice": "음성",
"voicePlaceholder": "음성 ID (선택 사항)",
"input": "텍스트",
"inputPlaceholder": "합성할 텍스트를 입력하세요...",
"instructions": "지침",
"instructionsPlaceholder": "예: 천천히 따뜻한 어조로 말하기",
"instructionsHint": "선택 사항인 말하기 지침입니다. 지원 여부는 백엔드에 따라 다릅니다."
},
"actions": {
"generate": "생성",
"generating": "생성 중..."
},
"empty": "생성된 오디오가 여기에 표시됩니다",
"toasts": {
"noText": "텍스트를 입력해 주세요",
"noModel": "모델을 선택해 주세요",
"generateFailed": "생성에 실패했습니다"
}
},
"sound": {
"title": "사운드 생성",
"labels": {
"model": "모델",
"prompt": "프롬프트",
"promptPlaceholder": "생성할 사운드를 설명하세요...",
"duration": "길이 (초)",
"language": "언어",
"vocalLanguage": "보컬 언어",
"lyrics": "가사 (선택 사항)",
"lyricsPlaceholder": "보컬 생성을 위한 가사",
"advanced": "고급 설정",
"seed": "시드",
"seedPlaceholder": "랜덤"
},
"actions": {
"generate": "생성",
"generating": "생성 중..."
},
"empty": "생성된 오디오가 여기에 표시됩니다",
"toasts": {
"noPrompt": "프롬프트를 입력해 주세요",
"noModel": "모델을 선택해 주세요",
"generateFailed": "생성에 실패했습니다"
}
},
"talk": {
"title": "대화",
"subtitle": "실시간 음성 대화",
"actions": {
"start": "세션 시작",
"stop": "세션 종료",
"connecting": "연결 중...",
"muted": "음소거됨",
"mute": "음소거",
"unmute": "음소거 해제"
},
"labels": {
"model": "모델",
"voice": "음성",
"voicePlaceholder": "alloy",
"language": "언어",
"languagePlaceholder": "en",
"instructions": "지침",
"instructionsPlaceholder": "어시스턴트의 페르소나를 설정하세요..."
},
"status": {
"idle": "대기 중",
"connecting": "연결 중...",
"listening": "듣는 중...",
"speaking": "말하는 중...",
"ended": "세션이 종료되었습니다"
},
"toasts": {
"noModel": "먼저 모델을 선택하세요",
"connectFailed": "연결에 실패했습니다: {{message}}"
}
},
"history": {
"title": "기록",
"empty": "아직 기록이 없습니다",
"deleteEntry": "항목 삭제",
"clear": "기록 지우기",
"clearTitle": "모든 기록 지우기",
"clearMessage": "모든 기록 항목을 제거하시겠습니까? 이 작업은 되돌릴 수 없습니다.",
"clearConfirm": "지우기",
"cleared": "기록이 지워졌습니다"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/i18n/media.pt-BR.json
+464
View File
@@ -0,0 +1,464 @@
{
"studio": {
"tabs": {
"images": "Imagens",
"video": "Vídeo",
"tts": "TTS",
"sound": "Som",
"transform": "Transformar",
"threed": "3D",
"overview": "Visão geral"
},
"overview": {
"eyebrow": "{{ready}} de {{total}} modalidades prontas",
"title": "Studio",
"subtitle": "Gere imagens, vídeos, 3D, fala e som com os modelos desta máquina.",
"canMake": "O que você pode criar",
"running": "Em execução agora",
"recent": "Saídas recentes",
"noModel": "Nenhum modelo instalado",
"install": "Instalar um modelo",
"ready": "Pronto",
"seconds": "{{seconds}}s",
"describe": {
"images": "Texto para imagem, imagem para imagem, imagens de referência",
"video": "Texto para vídeo e imagem para vídeo",
"threed": "Reconstrução de malha a partir de imagem",
"tts": "Texto para fala usando sua biblioteca de vozes",
"sound": "Música e efeitos sonoros a partir de um prompt",
"transform": "Separação, aprimoramento e conversão de voz"
}
},
"groups": {
"create": "Criar",
"voice": "Voz",
"transform": "Transformar"
}
},
"image": {
"title": "Geração de Imagens",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Descreva a imagem que deseja gerar...",
"negativePrompt": "Prompt Negativo",
"negativePromptPlaceholder": "O que evitar...",
"size": "Tamanho",
"count": "Quantidade (1-4)",
"advanced": "Configurações Avançadas",
"imageInputs": "Entradas de Imagem",
"steps": "Etapas",
"stepsPlaceholder": "20",
"seed": "Semente",
"seedPlaceholder": "Aleatória",
"sourceImage": "Imagem de Origem (img2img)",
"refImages": "Imagens de Referência",
"refImagesAdded_one": "{{count}} imagem adicionada",
"refImagesAdded_other": "{{count}} imagens adicionadas"
},
"actions": {
"view": "Ver",
"generate": "Gerar",
"generating": "Gerando..."
},
"empty": "As imagens geradas aparecerão aqui",
"toasts": {
"noPrompt": "Digite um prompt",
"noModel": "Selecione um modelo",
"noResults": "Nenhuma imagem gerada"
}
},
"video": {
"title": "Geração de Vídeo",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Descreva o vídeo que deseja gerar...",
"duration": "Duração (segundos)",
"fps": "FPS",
"size": "Tamanho",
"advanced": "Configurações Avançadas",
"seed": "Semente",
"seedPlaceholder": "Aleatória",
"frames": "Quadros",
"referenceMedia": "Mídia de referência",
"startImage": "Imagem inicial",
"endImage": "Imagem final",
"avatarAudio": "Áudio do avatar"
},
"actions": {
"generate": "Gerar",
"generating": "Gerando..."
},
"empty": "O vídeo gerado aparecerá aqui",
"toasts": {
"noPrompt": "Digite um prompt",
"noModel": "Selecione um modelo",
"noResults": "Nenhum vídeo gerado"
}
},
"threed": {
"title": "Geração 3D",
"labels": {
"model": "Modelo",
"image": "Imagem de entrada",
"quality": "Qualidade",
"quality_auto": "Automática (melhor disponível)",
"quality_coarse": "Pré-visualização grosseira (rápida)",
"quality_512": "512³ fino",
"quality_1024": "Cascata 1024³ (lenta)",
"background": "Fundo",
"background_auto": "Remover automaticamente o fundo sólido",
"background_keep": "Manter original",
"background_black": "Remover preto",
"background_white": "Remover branco",
"advanced": "Configurações Avançadas",
"steps": "Etapas de forma",
"textureSteps": "Etapas de material",
"guidance": "Orientação",
"seed": "Semente",
"seedPlaceholder": "Aleatória"
},
"actions": {
"generate": "Gerar",
"generating": "Gerando...",
"remesh": "Aplicar remalhamento",
"remeshing": "Aplicando remalhamento...",
"showOriginal": "Mostrar original",
"download": "Baixar GLB"
},
"remesh": {
"title": "Remalhamento hermético para impressão",
"detail": "Detalhe do remalhamento",
"coarser": "Mais grosseiro · mais rápido",
"finer": "Mais fino · mais lento",
"hint": "O detalhe controla as menores características preservadas. O deslocamento de envolvimento segue automaticamente.",
"ready": "Pré-visualizando o modelo remalhado hermético. Esta é a versão que será baixada."
},
"viewer": {
"wireframe": "Wireframe",
"autoRotate": "Rotação automática",
"noWebgl": "WebGL2 não está disponível neste navegador — baixe o GLB para vê-lo em outro lugar.",
"contextLost": "A visualização 3D perdeu o contexto da GPU — recarregue a página para restaurá-la.",
"stats": "{{verts}} vértices · {{tris}} triângulos",
"hint": "arrastar: girar · pinça/roda: zoom · dois dedos/clique direito: mover · duplo clique: redefinir"
},
"empty": "O modelo 3D gerado aparecerá aqui",
"toasts": {
"noImage": "Forneça uma imagem de entrada",
"noModel": "Selecione um modelo",
"noResults": "Nenhum modelo gerado"
}
},
"tts": {
"title": "Texto para Fala",
"labels": {
"model": "Modelo",
"voice": "Voz",
"voicePlaceholder": "Falante opcional ou ID de voz",
"input": "Texto",
"inputPlaceholder": "Digite o texto para sintetizar...",
"instructions": "Instruções",
"instructionsPlaceholder": "Por exemplo: Fale devagar e com calor",
"instructionsHint": "Orientações opcionais de interpretação. O suporte depende do backend."
},
"actions": {
"generate": "Gerar",
"generating": "Gerando..."
},
"empty": "O áudio gerado aparecerá aqui",
"toasts": {
"noText": "Digite o texto",
"noModel": "Selecione um modelo",
"generated": "Fala gerada",
"generateFailed": "Falha na geração"
},
"voiceLibrary": {
"cloningReady": "Clonagem de voz",
"loading": "Carregando vozes salvas…",
"modelDefault": "Usar padrão do modelo",
"empty": "Nenhuma voz salva ainda.",
"create": "Criar uma",
"manage": "Gerenciar Biblioteca de Vozes",
"namedVoiceHint": "Este modelo não aceita perfis de áudio de referência. Você ainda pode digitar um falante nomeado suportado pelo seu backend."
}
},
"voiceLibrary": {
"title": "Biblioteca de Vozes",
"subtitle": "Crie e gerencie vozes de referência reutilizáveis para cada modelo instalado que suporta clonagem de voz.",
"loading": "Carregando vozes salvas…",
"listLabel": "Perfis de voz salvos",
"status": {
"ready": "Pronto"
},
"summary": {
"label": "Status da biblioteca de vozes",
"profiles": "vozes salvas",
"modelsReady_one": "{{count}} modelo compatível pronto",
"modelsReady_other": "{{count}} modelos compatíveis prontos",
"noModels": "Nenhum modelo compatível instalado"
},
"search": {
"label": "Buscar perfis de voz",
"placeholder": "Buscar vozes, idiomas ou transcrições"
},
"filters": {
"language": "Filtrar por idioma",
"allLanguages": "Todos os idiomas"
},
"empty": {
"title": "Sua biblioteca de vozes está vazia",
"body": "Grave ou envie um clipe de referência autorizado, adicione a transcrição exata e reuse-o nos modelos de TTS compatíveis."
},
"noResults": {
"title": "Nenhuma voz corresponde a estes filtros",
"body": "Tente uma busca diferente ou mostre todos os idiomas."
},
"detail": {
"eyebrow": "Voz selecionada",
"emptyTitle": "Selecione uma voz para inspecioná-la",
"emptyBody": "Áudio de referência, transcrição, compatibilidade e detalhes de consentimento aparecerão aqui.",
"referenceAudio": "Áudio de referência",
"transcript": "Transcrição de referência"
},
"metadata": {
"language": "Idioma",
"languageUnknown": "Não especificado",
"duration": "Duração",
"sampleRate": "Taxa de amostragem",
"created": "Criado em"
},
"consent": {
"confirmed": "Consentimento confirmado",
"confirmedAt": "Confirmado em {{date}}"
},
"modelSetup": {
"title": "Instalar um modelo de clonagem de voz",
"body": "Esta voz está pronta, mas precisa de um modelo que aceite áudio de referência. Escolha um verificado por este servidor.",
"loading": "Encontrando modelos compatíveis em suas galerias…",
"backendUnknown": "Backend selecionado durante a instalação",
"install": "Instalar",
"installing": "Instalando…",
"loadFailed": "Recomendações compatíveis não puderam ser carregadas.",
"noneAvailable": "Nenhum modelo compatível não instalado está disponível nas galerias configuradas.",
"capabilityNote": "As recomendações vêm dos metadados de capacidade do modelo, não de uma lista de modelos do lado do navegador."
},
"api": {
"title": "Uso da API e modelos compatíveis",
"summary": "Use esta voz fora do navegador ou verifique o suporte do modelo",
"body": "Envie a URI de voz estável do perfil com cada requisição de fala. O LocalAI resolve o áudio de referência privado e a transcrição no momento da geração; o YAML do modelo não é modificado.",
"compatibleModels": "Modelos compatíveis instalados",
"noInstalledModels": "Nenhum ainda. Instale um dos modelos verificados pelo servidor acima antes de enviar esta requisição.",
"curlExample": "Exemplo com cURL",
"copy": "Copiar requisição",
"endpointNote": "Os mesmos campos de modelo, entrada e voz também funcionam com POST /tts."
},
"actions": {
"create": "Criar voz",
"createFirst": "Crie sua primeira voz",
"retry": "Tentar novamente",
"clearFilters": "Limpar filtros",
"useInTTS": "Usar em Texto para Fala",
"delete": "Excluir voz",
"installModel": "Instalar um modelo compatível",
"browseModels": "Explorar todos os modelos"
},
"deleteDialog": {
"title": "Excluir esta voz?",
"message": "“{{name}}” e seu áudio de referência serão removidos permanentemente. O YAML dos modelos existentes não é alterado.",
"deleting": "Excluindo…"
},
"toasts": {
"deleted": "Voz {{name}} excluída",
"installStarted": "Instalando {{name}}. O progresso está disponível em Operações.",
"installFailed": "Não foi possível iniciar a instalação: {{message}}",
"apiCopied": "Requisição de API copiada",
"copyFailed": "Não foi possível copiar a requisição de API"
}
},
"voiceCreate": {
"eyebrow": "Biblioteca de Vozes",
"title": "Criar uma voz reutilizável",
"subtitle": "Adicione um clipe de referência limpo e sua transcrição exata. O LocalAI cuida dos parâmetros de clonagem específicos do backend no momento da geração.",
"sections": {
"reference": {
"title": "Áudio de referência",
"body": "Grave no navegador ou envie um clipe existente. A fala deve ser limpa, natural e de um único falante."
},
"details": {
"title": "Identidade e transcrição",
"body": "Dê aos administradores um nome reconhecível e transcreva o clipe exatamente, incluindo hesitações e pontuação."
},
"consent": {
"title": "Autorização",
"body": "A clonagem de voz pode ser sensível. Confirme a permissão antes que esta referência seja salva."
}
},
"audio": {
"label": "Referência de voz",
"normalizing": "Preparando um PCM WAV seguro…",
"preview": "Referência normalizada",
"durationError": "O áudio de referência deve ter entre 1 segundo e 2 minutos.",
"decodeError": "Este navegador não conseguiu decodificar o arquivo de áudio selecionado.",
"qualityReady": "Duração recomendada",
"qualityHint": "6–30 segundos geralmente produzem um clone mais forte"
},
"fields": {
"name": "Nome da voz",
"namePlaceholder": "ex.: Narrador de documentário",
"language": "Idioma (opcional)",
"languagePlaceholder": "ex.: pt-BR",
"description": "Descrição (opcional)",
"descriptionPlaceholder": "Tom, origem ou uso pretendido",
"transcript": "Transcrição exata",
"transcriptPlaceholder": "Digite cada palavra falada no clipe de referência…",
"transcriptHint": "Corresponda à gravação exatamente. Não reescreva a gramática nem remova pausas e hesitações."
},
"consent": {
"title": "Confirmo que esta voz pode ser clonada",
"body": "Tenho a permissão do falante ou outra base legal para armazenar e usar esta gravação para fala sintetizada."
},
"readiness": {
"title": "Prontidão do perfil",
"body": "Uma voz fica disponível em Texto para Fala quando todos os itens obrigatórios estão completos.",
"audio": "Referência PCM-WAV válida",
"quality": "Janela de qualidade de 6–30 segundos",
"name": "Nome de voz reconhecível",
"transcript": "Transcrição de referência exata",
"consent": "Autorização confirmada"
},
"privacy": {
"title": "Privada por padrão. ",
"body": "O áudio de referência é armazenado no diretório de dados do LocalAI com permissões de arquivo restritas e nunca é exposto como um caminho de pasta de modelo."
},
"actions": {
"back": "Voltar para a biblioteca",
"cancel": "Cancelar",
"save": "Salvar voz",
"saving": "Salvando…"
},
"toasts": {
"created": "{{name}} está pronta para uso"
}
},
"sound": {
"title": "Geração de Som",
"labels": {
"model": "Modelo",
"prompt": "Prompt",
"promptPlaceholder": "Descreva o som que deseja gerar...",
"instrumental": "Instrumental",
"bpm": "BPM",
"duration": "Duração (segundos)",
"language": "Idioma",
"timesignature": "Fórmula de Compasso",
"keyscale": "Tom/Escala",
"vocalLanguage": "Idioma vocal",
"vocalLanguagePlaceholder": "ex.: Português",
"caption": "Legenda",
"lyrics": "Letra",
"lyricsPlaceholder": "Letra para geração vocal",
"simple": "Simples",
"advanced": "Avançado",
"seed": "Semente",
"seedPlaceholder": "Aleatória",
"thinkMode": "Modo de pensamento"
},
"actions": {
"generate": "Gerar",
"generating": "Gerando..."
},
"empty": "O áudio gerado aparecerá aqui",
"toasts": {
"noPrompt": "Digite um prompt",
"noModel": "Selecione um modelo",
"generateFailed": "Falha na geração"
}
},
"audioTransform": {
"title": "Transformação de Áudio",
"labels": {
"model": "Modelo",
"audio": "Áudio (obrigatório)",
"reference": "Referência (opcional)",
"referenceHelp": "Sinal de loopback / extremidade distante para cancelamento de eco, falante alvo para conversão de voz. Deixe vazio para transformação incondicional.",
"advancedParameters": "Parâmetros avançados",
"advancedParametersHelp": "específicos do backend (uma chave=valor por linha, ex.: noise_gate=true)",
"advancedParametersPlaceholder": "# Opcional. Para LocalVQE:\n# noise_gate=true\n# noise_gate_threshold_dbfs=-50"
},
"input": {
"upload": "Enviar",
"uploadDescription": "Solte um arquivo aqui ou",
"uploadBrowse": "navegue",
"record": "Gravar",
"startRecording": "Iniciar gravação",
"stop": "Parar",
"encoding": "Codificando...",
"clear": "Limpar",
"microphoneUnavailable": "A captura de microfone não está disponível neste navegador.",
"echoNotice": "Os navegadores costumam aplicar seu próprio cancelamento de eco WebRTC e supressão de ruído por padrão. Isso geralmente resulta em desempenho pior do que executar o LocalVQE no áudio bruto.",
"echoTest": "Teste de eco (grave o microfone enquanto reproduz a referência)",
"stopEchoTest": "Parar teste de eco"
},
"result": {
"audio": "Áudio",
"reference": "Referência",
"inputSpectrum": "Espectro de entrada",
"outputSpectrum": "Espectro de saída",
"outputSpectrumHint": "Transforme para comparar a atenuação",
"output": "Saída"
},
"actions": {
"transform": "Transformar",
"processing": "Processando..."
},
"empty": "Escolha um arquivo de áudio (e referência opcional) para transformar"
},
"talk": {
"title": "Falar",
"subtitle": "Conversa de voz em tempo real",
"actions": {
"start": "Iniciar sessão",
"stop": "Encerrar sessão",
"connecting": "Conectando...",
"muted": "Mudo",
"mute": "Silenciar",
"unmute": "Reativar som"
},
"labels": {
"model": "Modelo",
"voice": "Voz",
"voicePlaceholder": "alloy",
"language": "Idioma",
"languagePlaceholder": "pt",
"instructions": "Instruções",
"instructionsPlaceholder": "Defina a persona do assistente..."
},
"status": {
"idle": "Ocioso",
"connecting": "Conectando...",
"listening": "Ouvindo...",
"speaking": "Falando...",
"ended": "Sessão encerrada"
},
"toasts": {
"noModel": "Selecione um modelo primeiro",
"connectFailed": "Falha ao conectar: {{message}}"
}
},
"history": {
"title": "Histórico",
"empty": "Nenhum histórico ainda",
"deleteEntry": "Excluir entrada",
"clear": "Limpar histórico",
"clearTitle": "Limpar todo o histórico",
"clearMessage": "Remover todas as entradas do histórico? Esta ação não pode ser desfeita.",
"clearConfirm": "Limpar",
"cleared": "Histórico limpo"
},
"request": {
"heading": "Requisição",
"copyCurl": "Copiar como curl",
"copied": "Copiado"
}
}
addons/studio/ui/src/i18n/media.zh-CN.json
+195
View File
@@ -0,0 +1,195 @@
{
"studio": {
"tabs": {
"images": "图像",
"video": "视频",
"tts": "TTS",
"sound": "声音",
"transform": "Transform",
"overview": "Overview"
},
"overview": {
"eyebrow": "{{ready}} of {{total}} modalities ready",
"title": "Studio",
"subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
"canMake": "What you can make",
"running": "Running now",
"recent": "Recent outputs",
"noModel": "No model installed",
"install": "Install a model",
"ready": "Ready",
"seconds": "{{seconds}}s",
"describe": {
"images": "Text to image, image to image, reference images",
"video": "Text to video and image to video",
"threed": "Image to mesh reconstruction",
"tts": "Text to speech using your voice library",
"sound": "Music and sound effects from a prompt",
"transform": "Separation, enhancement and voice conversion"
}
},
"groups": {
"create": "Create",
"voice": "Voice",
"transform": "Transform"
}
},
"image": {
"title": "图像生成",
"labels": {
"model": "模型",
"prompt": "提示词",
"promptPlaceholder": "描述您想要生成的图像...",
"negativePrompt": "负面提示词",
"negativePromptPlaceholder": "想要避免的内容...",
"size": "尺寸",
"count": "数量 (1-4)",
"advanced": "高级设置",
"imageInputs": "图像输入",
"steps": "步数",
"stepsPlaceholder": "20",
"seed": "随机种子",
"seedPlaceholder": "随机",
"sourceImage": "源图像 (img2img)",
"refImages": "参考图像",
"refImagesAdded_one": "已添加 {{count}} 张图像",
"refImagesAdded_other": "已添加 {{count}} 张图像"
},
"actions": {
"view": "View",
"generate": "生成",
"generating": "生成中..."
},
"empty": "生成的图像将显示在此处",
"toasts": {
"noPrompt": "请输入提示词",
"noModel": "请选择一个模型",
"noResults": "未生成图像"
}
},
"video": {
"title": "视频生成",
"labels": {
"model": "模型",
"prompt": "提示词",
"promptPlaceholder": "描述您想要生成的视频...",
"duration": "时长（秒）",
"fps": "FPS",
"size": "尺寸",
"advanced": "高级设置",
"seed": "随机种子",
"seedPlaceholder": "随机",
"frames": "帧数",
"referenceMedia": "参考媒体",
"startImage": "起始图像",
"endImage": "结束图像",
"avatarAudio": "虚拟形象音频"
},
"actions": {
"generate": "生成视频",
"generating": "生成中..."
},
"empty": "生成的视频将显示在此处",
"toasts": {
"noPrompt": "请输入提示词",
"noModel": "请选择一个模型",
"noResults": "未生成视频"
}
},
"tts": {
"title": "文字转语音",
"labels": {
"model": "模型",
"voice": "声音",
"voicePlaceholder": "可选的声音 ID",
"input": "文本",
"inputPlaceholder": "输入要合成的文本...",
"instructions": "朗读指令",
"instructionsPlaceholder": "例如：缓慢、温暖地朗读",
"instructionsHint": "可选的表达方式指导。是否支持取决于后端。"
},
"actions": {
"generate": "生成音频",
"generating": "生成中..."
},
"empty": "生成的音频将显示在此处",
"toasts": {
"noText": "请输入文本",
"noModel": "请选择一个模型",
"generateFailed": "生成失败"
}
},
"sound": {
"title": "声音生成",
"labels": {
"model": "模型",
"prompt": "提示词",
"promptPlaceholder": "描述您想要生成的声音...",
"duration": "时长（秒）",
"language": "语言",
"vocalLanguage": "声乐语言",
"lyrics": "歌词（可选）",
"lyricsPlaceholder": "用于声乐生成的歌词",
"advanced": "高级设置",
"seed": "随机种子",
"seedPlaceholder": "随机"
},
"actions": {
"generate": "生成",
"generating": "生成中..."
},
"empty": "生成的音频将显示在此处",
"toasts": {
"noPrompt": "请输入提示词",
"noModel": "请选择一个模型",
"generateFailed": "生成失败"
}
},
"talk": {
"title": "通话",
"subtitle": "实时语音对话",
"actions": {
"start": "开始会话",
"stop": "结束会话",
"connecting": "连接中...",
"muted": "已静音",
"mute": "静音",
"unmute": "取消静音"
},
"labels": {
"model": "模型",
"voice": "声音",
"voicePlaceholder": "alloy",
"language": "语言",
"languagePlaceholder": "zh",
"instructions": "指令",
"instructionsPlaceholder": "设置助手的角色..."
},
"status": {
"idle": "空闲",
"connecting": "连接中...",
"listening": "正在聆听...",
"speaking": "正在说话...",
"ended": "会话已结束"
},
"toasts": {
"noModel": "请先选择一个模型",
"connectFailed": "连接失败：{{message}}"
}
},
"history": {
"title": "历史",
"empty": "暂无历史",
"deleteEntry": "删除条目",
"clear": "清除历史",
"clearTitle": "清除全部历史",
"clearMessage": "删除所有历史条目？此操作无法撤销。",
"clearConfirm": "清除",
"cleared": "历史已清除"
},
"request": {
"heading": "Request",
"copyCurl": "Copy as curl",
"copied": "Copied"
}
}
addons/studio/ui/src/index.jsx
+20
View File
@@ -0,0 +1,20 @@
// The entry of the Studio add-on. The pages are the ones that used to be core
// routes; they keep the host modules they were written against (React, the
// router, i18next, the shared components), and the stand-ins in
// addons/internal/host-compat answer those through the SDK. The page code is
// loaded after the SDK is set, because a stand-in reads the SDK when it starts.
import { requireHost, setSdk } from '../../../internal/host-compat/src/runtime.js'
// What the pages need from the host beyond SDK 2's first set.
const NEEDS = [
'ui.LoadingSpinner', 'ui.Modal', 'ui.Lightbox', 'ui.ModelSelector', 'ui.PageHeader', 'ui.ConfirmDialog',
'ui.MediaInput', 'ui.WaveformPlayer', 'hooks.useModels', 'hooks.useResources', 'hooks.useVoiceProfiles',
'hooks.useOperations', 'hooks.useMediaCapture', 'hooks.useObjectUrl', 'navigation.openApp',
]
export async function createAddon(sdk) {
requireHost(sdk, NEEDS)
setSdk(sdk)
const { createStudio } = await import('./addon.jsx')
return createStudio(sdk)
}
addons/studio/ui/src/pages/AudioTransform.jsx
+549
View File
@@ -0,0 +1,549 @@
import { useState, useEffect, useRef } from 'react'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
import { useParams, useOutletContext } from 'react-router-dom'
import { CAP_AUDIO_TRANSFORM } from '../utils/capabilities'
// eslint-disable-next-line no-unused-vars
import WaveformPlayer from '../components/audio/WaveformPlayer'
// eslint-disable-next-line no-unused-vars
import Spectrogram from '../components/audio/Spectrogram'
import { audioTransformApi, fetchUrl } from '../utils/api'
import { useMediaCapture } from '../hooks/useMediaCapture'
import useObjectUrl from '../hooks/useObjectUrl'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff, useHandoffSource, blobToFile } from '../hooks/useStudioHandoff'
// eslint-disable-next-line no-unused-vars
import HandoffNote from '../components/studio/HandoffNote'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ModelChip, Field, Fold, RunArea, JobCard, FailedCard, ResultCard, ViewCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
import { useTranslation } from 'react-i18next'
import Icon from '../components/Icon'
// AudioTransform — Studio tab for the audio_transform capability. Takes a
// primary audio file plus an optional reference (loopback for AEC, target
// speaker for voice conversion, etc.) and shows three synchronized
// waveforms: input audio / reference / output. Supports both file upload
// and direct mic recording, plus an "echo test" mode that records mic
// while playing the reference — the recorded mic picks up the speaker
// bleed of the reference, giving the user a real (mic, ref) pair to test
// echo cancellation against.
export default function AudioTransform() {
const { t } = useTranslation('media')
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
// Opened from the Studio front page, the audio it was made from is the input.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const [audioFile, setAudioFile] = useState(null)
const [referenceFile, setReferenceFile] = useState(null)
const [outputUrl, setOutputUrl] = useState(null)
const [paramsText, setParamsText] = useState('')
const [showAdvanced, setShowAdvanced] = useState(false)
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
// What was actually sent, so the panel records rather than predicts.
const [lastRequest, setLastRequest] = useState(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('audio-transform')
const ws = useWorkspace({ type: 'transform', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
const wantsSource = handoff.edge === 'transform' || handoff.edge === 'take'
const source = useHandoffSource(handoff, wantsSource)
useEffect(() => {
if (source.status !== 'ready') return
const base = (source.item?.url || '').split('?')[0].split('/').pop() || 'audio.wav'
setAudioFile(blobToFile(source.blob, base))
}, [source])
// Hidden <audio> element that plays the reference out the speakers while
// the mic records — the recording captures the user's voice plus the
// speaker-bleed echo. Headphones short-circuit the path; document only.
const echoAudioRef = useRef(null)
const echoCap = useMediaCapture('audio')
const echoActive = echoCap.active || echoCap.recording
// Blob URLs derived from File state. useObjectUrl revokes the previous
// URL when its source changes and on unmount, so the cleanup is correct
// without a separate effect tracking each setter.
const audioUrl = useObjectUrl(audioFile)
const referenceUrl = useObjectUrl(referenceFile)
useEffect(() => {
return () => { if (outputUrl) URL.revokeObjectURL(outputUrl) }
}, [outputUrl])
const activeEntry = selectedEntry || (lastId ? historyProps.entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const parseParams = () => {
const out = {}
for (const raw of paramsText.split('\n')) {
const line = raw.trim()
if (!line || line.startsWith('#')) continue
const eq = line.indexOf('=')
if (eq < 0) continue
const k = line.slice(0, eq).trim()
const v = line.slice(eq + 1).trim()
if (k) out[k] = v
}
return out
}
const run = async () => {
if (!model) { addToast(t('studio.workspace.transform.noModel'), 'warning'); return }
if (!audioFile) { addToast(t('studio.workspace.transform.noAudio'), 'warning'); return }
setLoading(true)
setError(null)
setLastId(null)
if (outputUrl) { URL.revokeObjectURL(outputUrl); setOutputUrl(null) }
// The audio itself is multipart, not JSON, so the panel records the fields
// that shape the request rather than the bytes.
setLastRequest({ model, format: 'wav', params: parseParams() })
try {
const { blob, serverUrl, inputUrl, referenceUrl: refServerUrl } = await audioTransformApi.process({
model,
audioFile,
referenceFile,
format: 'wav',
params: parseParams(),
})
const url = URL.createObjectURL(blob)
setOutputUrl(url)
addToast(t('studio.workspace.transform.done'), 'success')
if (serverUrl) {
// Save the persisted (input, reference, output) triple so a click
// in the History panel can later replay all three players. The
// server held onto the converted 16 kHz mono inputs — saving raw
// upload bytes in localStorage would blow past quota in a few runs.
setLastId(addEntry({
prompt: describeRun(audioFile, referenceFile),
model,
params: parseParams(),
results: [
{ kind: 'output', url: serverUrl },
inputUrl ? { kind: 'input', url: inputUrl } : null,
refServerUrl ? { kind: 'reference', url: refServerUrl } : null,
].filter(Boolean),
parentId: handoff.from || undefined,
edge: handoff.edge || undefined,
}))
}
selectEntry(null)
ws.end()
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
// Re-run with edits: the model and the parameters go back into the form, and
// so do the audio and reference the server kept for that run.
const rerun = async (it) => {
const entry = historyProps.entries.find(e => e.id === it.id)
if (!entry) return
const next = {
model: entry.model || model,
params: Object.entries(entry.params || {}).map(([k, v]) => `${k}=${v}`).join('\n'),
}
setModel(next.model)
setParamsText(next.params)
if (next.params) setShowAdvanced(true)
selectEntry(null)
ws.begin(next)
const fetchFile = async (url, fallback) => {
const res = await fetchUrl(url)
if (!res.ok) throw new Error(`HTTP ${res.status}`)
return blobToFile(await res.blob(), url.split('?')[0].split('/').pop() || fallback)
}
const input = entry.results?.find(r => r.kind === 'input')
const reference = entry.results?.find(r => r.kind === 'reference')
try {
if (input) setAudioFile(await fetchFile(input.url, 'audio.wav'))
setReferenceFile(reference ? await fetchFile(reference.url, 'reference.wav') : null)
} catch {
addToast(t('studio.workspace.transform.inputGone'), 'warning')
}
}
// The echo-test playback listener is held in a ref so stopEchoTest can
// detach it without depending on the closure that registered it.
const echoEndedListenerRef = useRef(null)
const detachEchoEndedListener = () => {
const audio = echoAudioRef.current
const listener = echoEndedListenerRef.current
if (audio && listener) audio.removeEventListener('ended', listener)
echoEndedListenerRef.current = null
}
const startEchoTest = async () => {
if (!referenceUrl) {
addToast(t('studio.workspace.transform.loadReference'), 'warning')
return
}
if (!echoCap.supported) {
addToast(t('studio.workspace.transform.noMic'), 'warning')
return
}
try {
// Acquire the mic first so the recording covers the entire ref playback.
await echoCap.start()
const recPromise = echoCap.startRecording()
const audio = echoAudioRef.current
if (audio) {
audio.currentTime = 0
const onEnded = () => {
detachEchoEndedListener()
echoCap.stopRecording()
}
echoEndedListenerRef.current = onEnded
audio.addEventListener('ended', onEnded)
try { await audio.play() } catch (_) { /* user-gesture gate, ignore */ }
}
const result = await recPromise
detachEchoEndedListener()
echoCap.stop()
const file = new File([result.blob], 'mic-echo-test.wav', { type: 'audio/wav' })
setAudioFile(file)
addToast(t('studio.workspace.transform.recorded'), 'success')
} catch (err) {
detachEchoEndedListener()
addToast(t('studio.workspace.transform.echoFailed', { message: err?.message || String(err) }), 'error')
}
}
const stopEchoTest = () => {
detachEchoEndedListener()
echoCap.stopRecording()
if (echoAudioRef.current) {
try { echoAudioRef.current.pause() } catch (_) { /* ignore */ }
}
echoCap.stop()
}
const labels = { model: t('studio.workspace.fields.model'), params: t('studio.workspace.fields.params') }
const changes = ws.changes({ model, params: paramsText }, labels)
const changed = new Set(changes.map(c => c.field))
const installed = ws.installed.byType.transform
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.transform') })
: !audioFile ? t('studio.workspace.transform.whyAudio') : ''
const paramCount = Object.keys(parseParams()).length
return (
<Workspace type="transform">
<ComposeCard
ws={ws}
icon="waveform"
title={t('audioTransform.title')}
lede={t('studio.workspace.lede.transform')}
onSubmit={(e) => { e.preventDefault(); run() }}
handoff={<HandoffNote source={source} handoff={handoff} wanted={wantsSource} onClear={() => setAudioFile(null)} />}
model={model}
noModel={noModel ? { type: 'transform', label: t('studio.tabs.transform'), onChanged: ws.installed.refetch } : null}
options={<ModelChip value={model} onChange={setModel} capability={CAP_AUDIO_TRANSFORM} changed={changed.has('model')} />}
fold={
<Fold
label={t('audioTransform.labels.advancedParameters')}
summary={foldSummary(paramCount ? [t('studio.workspace.transform.paramCount', { count: paramCount })] : [], [t('audioTransform.labels.advancedParametersHelp')])}
open={showAdvanced}
onToggle={() => setShowAdvanced(v => !v)}
id="transform-advanced-options"
>
<Field label={t('audioTransform.labels.advancedParameters')} htmlFor="transform-params" changed={changed.has('params')} hint={t('audioTransform.labels.advancedParametersHelp')}>
<textarea
id="transform-params"
className="textarea"
value={paramsText}
onChange={(e) => setParamsText(e.target.value)}
placeholder={t('audioTransform.labels.advancedParametersPlaceholder')}
rows={4}
/>
</Field>
</Fold>
}
changes={changes}
submit={{ label: t('audioTransform.actions.transform'), busyLabel: t('audioTransform.actions.processing'), busy: loading, disabled: !model || !audioFile || noModel, why }}
>
<div className="ws-inputs">
<AudioInput
label={t('audioTransform.labels.audio')}
file={audioFile}
onChange={setAudioFile}
/>
<AudioInput
label={t('audioTransform.labels.reference')}
help={t('audioTransform.labels.referenceHelp')}
file={referenceFile}
onChange={setReferenceFile}
/>
</div>
{referenceFile && (
<div className="audio-transform-echo">
<p className="audio-transform-echo__notice" role="note">
<Icon name="info" />
<span>
{t('audioTransform.input.echoNotice')}
</span>
</p>
<div className="audio-transform-echo__row">
<button
type="button"
className={`dk-btn ${echoActive ? 'dk-btn--secondary' : 'dk-btn--primary'} dk-btn--sm`}
onClick={echoActive ? stopEchoTest : startEchoTest}
>
{echoActive
? <><Icon name="stop" /> {t('audioTransform.input.stopEchoTest')}</>
: <><Icon name="headphones" /> {t('audioTransform.input.echoTest')}</>}
</button>
{echoActive && echoCap.recording && (
<span className="audio-transform-echo__elapsed">
{t('studio.workspace.transform.recording', { seconds: echoCap.elapsed.toFixed(1) })}
</span>
)}
</div>
{/* Hidden player for the reference clip during the echo test.
Hidden because the user already has the WaveformPlayer in
the preview pane — this is just the audible source. */}
<audio ref={echoAudioRef} src={referenceUrl} preload="auto" hidden />
</div>
)}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('audioTransform.actions.processing')} detail={[model, audioFile?.name].filter(Boolean).join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : selectedEntry ? (
<ResultCard
ws={ws}
item={item}
title={selectedEntry.prompt}
meta={[selectedEntry.model]}
download={{ href: selectedEntry.results.find(r => r.kind === 'output')?.url, name: `audio-transform-${selectedEntry.model || 'output'}.wav` }}
onRerun={rerun}
>
<div className="ws-audio audio-transform-stack">
{selectedEntry.results.map((r) => (
<WaveformPlayer
key={r.kind || r.url}
src={r.url}
label={resultLabel(r)}
height={r.kind === 'output' ? 120 : 96}
dimmed={r.kind === 'reference'}
/>
))}
</div>
</ResultCard>
) : outputUrl ? (
<ResultCard
ws={ws}
item={item}
title={describeRun(audioFile, referenceFile)}
meta={[item?.model || model]}
download={{ href: outputUrl, name: `audio-transform-${model || 'output'}-${new Date().toISOString().slice(0, 10)}.wav` }}
onRerun={rerun}
>
<div className="ws-audio audio-transform-stack">
<div className="audio-spectrogram-pair">
<Spectrogram src={audioUrl} label={t('audioTransform.result.inputSpectrum')} testId="spectrogram-input" />
<Spectrogram src={outputUrl} label={t('audioTransform.result.outputSpectrum')} testId="spectrogram-output" />
</div>
<WaveformPlayer src={audioUrl} label={t('audioTransform.result.audio')} height={96} />
<WaveformPlayer src={referenceUrl} label={t('audioTransform.result.reference')} height={96} dimmed={!referenceFile} />
<WaveformPlayer src={outputUrl} label={t('audioTransform.result.output')} height={120} />
</div>
</ResultCard>
) : audioUrl ? (
<ViewCard title={t('audioTransform.result.audio')} sub={t('studio.workspace.transform.waiting')}>
<div className="ws-audio audio-transform-stack">
<div className="audio-spectrogram-pair">
<Spectrogram src={audioUrl} label={t('audioTransform.result.inputSpectrum')} testId="spectrogram-input" />
<div className="audio-spectrogram">
<div className="audio-spectrogram__label">{t('audioTransform.result.outputSpectrum')}</div>
<div className="audio-spectrogram__canvas-wrap audio-spectrogram__canvas-wrap--empty">
<span className="audio-spectrogram__hint">{t('audioTransform.result.outputSpectrumHint')}</span>
</div>
</div>
</div>
<WaveformPlayer src={audioUrl} label={t('audioTransform.result.audio')} height={96} />
<WaveformPlayer src={referenceUrl} label={t('audioTransform.result.reference')} height={96} dimmed={!referenceFile} />
</div>
</ViewCard>
) : (
<EmptyRun icon="waveform" text={t('audioTransform.empty')} />
)}
<RequestPanel endpoint="/v1/audio/transform" body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={activeEntry?.id}
onSelect={historyProps.onSelect}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
</Workspace>
)
}
function describeRun(audioFile, referenceFile) {
const parts = []
if (audioFile?.name) parts.push(`audio: ${audioFile.name}`)
if (referenceFile?.name) parts.push(`reference: ${referenceFile.name}`)
return parts.join(' + ') || 'audio transform'
}
function resultLabel(r) {
switch (r.kind) {
case 'input': return 'Audio'
case 'reference': return 'Reference'
case 'output': return 'Output'
default: return ''
}
}
// AudioInput — drag-drop / file-pick for an audio file, with an inline
// mic-record tab. Emits a single File via onChange (recordings are wrapped
// as `File([blob], 'recording-XXX.wav', { type: 'audio/wav' })` so callers
// can treat them identically to uploaded files).
// eslint-disable-next-line no-unused-vars
function AudioInput({ label, help, file, onChange }) {
const { t } = useTranslation('media')
const [tab, setTab] = useState('upload') // 'upload' | 'record'
const cap = useMediaCapture('audio')
const [recordPending, setRecordPending] = useState(false)
const [hover, setHover] = useState(false)
const onDrop = (e) => {
e.preventDefault()
setHover(false)
const f = e.dataTransfer.files?.[0]
if (f) onChange(f)
}
const onPick = (e) => {
const f = e.target.files?.[0]
if (f) onChange(f)
}
const startRecord = async () => {
await cap.start()
if (cap.error) return
setRecordPending(true)
try {
const promise = cap.startRecording()
if (!promise) return
const result = await promise
const stamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19)
onChange(new File([result.blob], `recording-${stamp}.wav`, { type: 'audio/wav' }))
} finally {
setRecordPending(false)
}
}
const stopRecord = () => cap.stopRecording()
const hasFile = !!file
return (
<div className="form-group">
<label className="form-label">{label}</label>
<div className="audio-transform-input">
<div className="audio-transform-input__tabs" role="tablist">
<button
type="button"
role="tab"
aria-selected={tab === 'upload'}
className={`audio-transform-input__tab${tab === 'upload' ? ' active' : ''}`}
onClick={() => setTab('upload')}
>
<Icon name="upload" /> {t('audioTransform.input.upload')}
</button>
<button
type="button"
role="tab"
aria-selected={tab === 'record'}
className={`audio-transform-input__tab${tab === 'record' ? ' active' : ''}`}
onClick={() => setTab('record')}
>
<Icon name="mic" /> {t('audioTransform.input.record')}
</button>
</div>
{tab === 'upload' && (
<div
className={`audio-transform-drop${hover ? ' audio-transform-drop--hover' : ''}`}
onDragEnter={(e) => { e.preventDefault(); setHover(true) }}
onDragOver={(e) => { e.preventDefault(); setHover(true) }}
onDragLeave={() => setHover(false)}
onDrop={onDrop}
>
{hasFile ? (
<div className="audio-transform-drop__file">
<Icon name="music" /> {file.name}
<button type="button" className="btn btn-secondary btn-sm" onClick={() => onChange(null)}>{t('audioTransform.input.clear')}</button>
</div>
) : (
<>
<Icon name="upload" /> {t('audioTransform.input.uploadDescription')}
<label className="audio-transform-drop__pick">
<input type="file" accept="audio/*" onChange={onPick} hidden />
{t('audioTransform.input.uploadBrowse')}
</label>
</>
)}
</div>
)}
{tab === 'record' && (
<div className="audio-transform-rec">
{!cap.supported && (
<div className="audio-transform-rec__notice">
<Icon name="info" /> {t('audioTransform.input.microphoneUnavailable')}
</div>
)}
{cap.supported && (
<>
{!cap.recording && !recordPending && (
<button type="button" className="btn btn-primary btn-sm" onClick={startRecord}>
<Icon name="circle" style={{ color: 'var(--color-error)' }} /> {t('audioTransform.input.startRecording')}
</button>
)}
{cap.recording && (
<button type="button" className="btn btn-secondary btn-sm" onClick={stopRecord}>
<Icon name="stop" /> {t('audioTransform.input.stop')} ({cap.elapsed.toFixed(1)}s)
</button>
)}
{recordPending && !cap.recording && (
<div className="audio-transform-rec__pending">{t('audioTransform.input.encoding')}</div>
)}
{cap.error && (
<div className="audio-transform-rec__notice audio-transform-rec__notice--error">
{cap.error}
</div>
)}
{hasFile && !cap.recording && (
<div className="audio-transform-drop__file mt-sm">
<Icon name="music" /> {file.name}
<button type="button" className="btn btn-secondary btn-sm" onClick={() => onChange(null)}>{t('audioTransform.input.clear')}</button>
</div>
)}
</>
)}
</div>
)}
</div>
{help && <div className="form-help">{help}</div>}
</div>
)
}
addons/studio/ui/src/pages/Diarization.jsx
+285
View File
@@ -0,0 +1,285 @@
// SPDX-License-Identifier: MIT
import { useEffect, useMemo, useRef, useState } from 'react'
// eslint-disable-next-line no-unused-vars
import { Link, useParams } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
// eslint-disable-next-line no-unused-vars
import Modal from '../components/Modal'
import { useAuth } from '../context/AuthContext'
import useObjectUrl from '../hooks/useObjectUrl'
import { CAP_DIARIZATION } from '../utils/capabilities'
import { diarizationApi, voiceApi } from '../utils/api'
import { rememberEnrollment } from '../utils/voiceEnrollments'
import { clock, hasText, runLength, speakerRows, toRttm, toSrt } from '../utils/diarization'
import { cssVars } from '../utils/modelLedger'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff, useHandoffSource, blobToFile } from '../hooks/useStudioHandoff'
import { useWorkspace } from '../hooks/useWorkspace'
// eslint-disable-next-line no-unused-vars
import HandoffNote from '../components/studio/HandoffNote'
// eslint-disable-next-line no-unused-vars
import Timeline from '../components/studio/Timeline'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ModelChip, SourceDrop, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import Icon from '../components/Icon'
// A text export built from the result in hand, handed to the browser as a file.
// eslint-disable-next-line no-unused-vars
function ExportButton({ label, build, name, type }) {
const save = () => {
const url = URL.createObjectURL(new Blob([build()], { type }))
const a = document.createElement('a')
a.href = url
a.download = name
a.click()
setTimeout(() => URL.revokeObjectURL(url), 1000)
}
return <button type="button" className="ws-tool ws-tool--text" onClick={save} data-testid={`export-${label.toLowerCase()}`}>{label}</button>
}
export default function Diarization() {
const { t } = useTranslation('media')
const text = (key, values) => t(`diarization.${key}`, values)
const { model: initialModel } = useParams()
const { hasFeature } = useAuth()
const canRemember = hasFeature('voice_recognition')
// Opened from the Studio front page, the audio it was made from is the input.
const handoff = useStudioHandoff()
const [model, setModel] = useState(initialModel || handoff.model || '')
const [file, setFile] = useState(null)
const [optIn, setOptIn] = useState(false)
const [result, setResult] = useState(null)
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [selected, setSelected] = useState(null)
const [name, setName] = useState('')
const [saving, setSaving] = useState(false)
const [saveError, setSaveError] = useState('')
const generation = useRef(0)
const saveLock = useRef(false)
const audio = useRef(null)
const end = useRef(null)
const timer = useRef(null)
const playback = useRef(0)
const url = useObjectUrl(file)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('diarization')
const ws = useWorkspace({ type: 'diarization', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
const wantsSource = handoff.edge === 'diarize'
const source = useHandoffSource(handoff, wantsSource)
useEffect(() => {
if (source.status !== 'ready') return
const base = (source.item?.url || '').split('?')[0].split('/').pop() || 'audio.wav'
setFile(blobToFile(source.blob, base))
}, [source])
function stop() {
playback.current++
clearTimeout(timer.current)
audio.current?.pause()
end.current = null
}
function invalidate() {
generation.current++
stop()
setResult(null); setSelected(null); setBusy(false); setSaving(false)
setError(''); setSaveError(''); saveLock.current = false; setLastId(null)
}
useEffect(() => {
const player = audio.current
return () => { playback.current++; clearTimeout(timer.current); player?.pause() }
}, [url])
useEffect(() => () => { generation.current++ }, [])
useEffect(() => {
if (!canRemember) { setOptIn(false); setSelected(null) }
}, [canRemember])
async function run() {
if (!file || !model || busy) return
invalidate()
selectEntry(null)
const token = generation.current
const requested = canRemember && optIn
setBusy(true)
try {
const data = await diarizationApi.run({ file, model, profiles: requested })
if (token !== generation.current) return
if (requested && !data.speaker_profiles) throw new Error(text('missingProfiles'))
// Keep the actual inference model with this export, not a later selection.
setResult({ ...data, inferenceModel: model, speaker_profiles: requested ? data.speaker_profiles : undefined, fileName: file.name })
// The front page lists this run. Only the file name, the model and two
// counts are kept: not the recording, not the transcript, no voice data.
const labels = new Set((data.segments || []).map(s => String(s.label ?? s.speaker)))
const speakers = data.speakers?.length || labels.size
const seconds = Math.round(Math.max(0, ...(data.segments || []).map(s => Number(s.end) || 0)))
setLastId(addEntry({ prompt: file.name, model, params: { speakers, seconds }, results: [], parentId: handoff.from || undefined, edge: handoff.edge || undefined }))
} catch (err) {
if (token === generation.current) setError(`${err.message}${requested ? ` ${text('unsupported')}` : ''}`)
} finally { if (token === generation.current) setBusy(false) }
}
async function preview(interval) {
stop()
const player = audio.current
if (!player) return
const token = generation.current
const playToken = playback.current
try {
player.currentTime = interval.start
end.current = interval.end
await player.play()
if (token !== generation.current || playToken !== playback.current) return
timer.current = setTimeout(stop, Math.max(0, interval.end - player.currentTime) * 1000)
} catch { if (token === generation.current) setError(text('previewError')) }
}
async function save(event) {
event.preventDefault()
if (!canRemember || !name.trim() || selected === null || saveLock.current || !result) return
const token = generation.current
const slot = selected
saveLock.current = true; setSaving(true); setSaveError('')
try {
const registered = await voiceApi.register({
model: result.inferenceModel, name: name.trim(), speaker_slot: slot,
speaker_profiles: result.speaker_profiles,
})
// A completed registration is real even if its recording is no longer open.
rememberEnrollment(registered)
if (token !== generation.current) return
const relabel = rows => rows?.map(row => String(row.label) === String(slot) ? { ...row, name: registered.name } : row)
setResult(current => ({ ...current, speakers: relabel(current.speakers), segments: relabel(current.segments) }))
setSelected(null)
} catch (err) { if (token === generation.current) setSaveError(err.message) }
finally { if (token === generation.current) { saveLock.current = false; setSaving(false) } }
}
// Raw labels are the only stable join key. Normalized IDs and array order can differ.
const rows = useMemo(() => speakerRows(result), [result])
const length = runLength(result)
const item = ws.itemById(selectedEntry?.id || lastId)
const installed = ws.installed.byType.diarization
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.diarization') })
: !file ? t('studio.workspace.diarization.whyFile') : ''
const showResult = !!result && !selectedEntry
const fileName = result?.fileName || file?.name || 'recording'
return (
<Workspace type="diarization">
<ComposeCard
ws={ws}
icon="users"
title={text('title')}
lede={text('subtitle')}
onSubmit={(e) => { e.preventDefault(); run() }}
handoff={<HandoffNote source={source} handoff={handoff} wanted={wantsSource} onClear={() => { invalidate(); setFile(null) }} />}
model={model}
noModel={noModel ? { type: 'diarization', label: t('studio.tabs.diarization'), onChanged: ws.installed.refetch } : null}
options={<span role="group" aria-label={text('model')} className="ws-chip-group">
<ModelChip value={model} capability={CAP_DIARIZATION} onChange={value => { if (value !== model) { invalidate(); setModel(value) } }} />
</span>}
submit={{ label: text('run'), busyLabel: text('running'), busy, disabled: !file || !model || noModel, why, icon: 'users' }}
>
<SourceDrop inputId="diarization-file" label={text('recording')} accept="audio/*,video/*" file={file} hint={t('studio.workspace.diarization.hint')} onFile={f => { invalidate(); setFile(f) }} />
{canRemember && <div className="ws-optin">
<label className="ws-check"><input type="checkbox" checked={optIn} onChange={e => { invalidate(); setOptIn(e.target.checked) }} /> <span>{text('optIn')}</span></label>
<p className="ws-hint">{text('warning')} <Link className="ws-link" to="/app/voice">{text('manage')}</Link></p>
</div>}
</ComposeCard>
<RunArea>
{busy ? (
<JobCard label={text('running')} detail={[model, file?.name].filter(Boolean).join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : showResult ? (
<ResultCard
ws={ws}
item={item}
title={fileName}
meta={[result.inferenceModel, t('studio.lineage.speakers', { count: rows.length }), clock(length), t('studio.workspace.diarization.segmentCount', { count: result.segments?.length || 0 })]}
actions={<>
<ExportButton label="RTTM" name={`${fileName.replace(/\.[^.]*$/, '')}.rttm`} type="text/plain" build={() => toRttm(result, fileName)} />
{hasText(result) && <ExportButton label="SRT" name={`${fileName.replace(/\.[^.]*$/, '')}.srt`} type="text/plain" build={() => toSrt(result)} />}
<ExportButton label="JSON" name={`${fileName.replace(/\.[^.]*$/, '')}.json`} type="application/json" build={() => JSON.stringify(result, null, 2)} />
</>}
onRerun={() => {}}
>
<div className="ws-diar">
{length > 0 && <Timeline rows={rows} length={length} />}
{url && <audio ref={audio} src={url} controls preload="metadata" className="ws-diar__audio" onTimeUpdate={() => { if (end.current !== null && audio.current.currentTime >= end.current) stop() }} />}
<div className="hstack ws-diar__head"><h3>{text('speakers')}</h3>
{canRemember && result.speaker_profiles && <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={stop}>{text('stop')}</button>}
</div>
<ul className="lanes ws-speakers">
{rows.map(row => {
const profile = result.speaker_profiles?.speakers.find(p => String(p.speaker) === String(row.label))
const usable = profile?.embedding?.length > 0 && !profile.unavailable_reason
return <li className="ws-speaker" key={row.label} data-testid={`speaker-${row.label}`} data-speaker={row.index % 6}>
<h4><i aria-hidden="true" />{row.name || row.id}</h4>
<p className="ws-speaker__talk">{t('studio.workspace.diarization.talk', { seconds: Math.round(row.seconds), percent: Math.round(row.share * 100) })}</p>
<span className="ws-speaker__bar" aria-hidden="true"><b style={cssVars({ '--w': `${Math.round(row.share * 100)}%` })} /></span>
{profile && canRemember && <>
<p>{text('duration', { seconds: profile.clean_duration })}</p>
<div className="hstack">{profile.intervals.map((interval, i) => <button key={i} type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={() => preview(interval)}>{text('preview', { number: i + 1 })}</button>)}</div>
{!usable && <p>{text('insufficient')} {profile.unavailable_reason && <small>({profile.unavailable_reason})</small>}</p>}
{!row.name && <button type="button" className="dk-btn dk-btn--primary dk-btn--sm" disabled={!usable} onClick={() => { stop(); setSelected(profile.speaker); setName(''); setSaveError('') }}>{text('nameAndRemember')}</button>}
</>}
</li>
})}
</ul>
<h3>{text('segments')}</h3>
<ol data-testid="segments" className="lanes ws-segments">
{result.segments?.map((segment, i) => {
const row = rows.find(r => r.label === String(segment.label ?? segment.speaker))
return <li key={segment.id ?? i} data-speaker={(row?.index ?? 0) % 6}>
<span className="ws-segments__time">{clock(segment.start)}</span>
<strong>{segment.name || segment.speaker}</strong>
<span className="ws-segments__range">{segment.start}–{segment.end}s</span>
{segment.text && <p>{segment.text}</p>}
</li>
})}
</ol>
</div>
</ResultCard>
) : selectedEntry ? (
<ResultCard
ws={ws}
item={item}
title={selectedEntry.prompt}
meta={[selectedEntry.model, selectedEntry.params?.speakers ? t('studio.lineage.speakers', { count: selectedEntry.params.speakers }) : '', selectedEntry.params?.seconds ? clock(selectedEntry.params.seconds) : '']}
onRerun={() => {}}
>
<p className="ws-quote">{t('studio.workspace.diarization.notKept')}</p>
</ResultCard>
) : (
<EmptyRun icon="users" text={t('studio.workspace.diarization.empty')} />
)}
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={item?.id}
onSelect={(id) => { if (result) invalidate(); historyProps.onSelect(id) }}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
{selected !== null && canRemember && <Modal ariaLabel={text('nameAndRemember')} onClose={() => { if (!saving) setSelected(null) }}>
<form className="stack" onSubmit={save} aria-label={text('nameAndRemember')}>
<h2>{text('nameAndRemember')}</h2>
<p>{text('warning')}</p>
<label className="form-label" htmlFor="diarization-name">{text('name')}</label>
<input className="input" id="diarization-name" required value={name} disabled={saving} onChange={e => setName(e.target.value)} />
{saveError && <p role="alert">{saveError}</p>}
<button type="submit" className="btn btn-primary" disabled={saving || !name.trim()}>{text(saving ? 'saving' : 'remember')}</button>
<button type="button" className="btn btn-secondary" disabled={saving} onClick={() => setSelected(null)}>{text('cancel')}</button>
</form>
</Modal>}
</Workspace>
)
}
addons/studio/ui/src/pages/ImageGen.jsx
+265
View File
@@ -0,0 +1,265 @@
import { useState, useEffect } from 'react'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
import { useParams, useOutletContext } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import { CAP_IMAGE } from '../utils/capabilities'
// eslint-disable-next-line no-unused-vars
import Lightbox from '../components/Lightbox'
import { imageApi, fileToBase64 } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff, useHandoffSource, blobToImageInput } from '../hooks/useStudioHandoff'
import { parseSize } from '../utils/studioWork'
// eslint-disable-next-line no-unused-vars
import HandoffNote from '../components/studio/HandoffNote'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ChipSelect, ModelChip, Field, Fold, SourceChip, Starters, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
import Icon from '../components/Icon'
const SIZES = ['256x256', '512x512', '768x768', '1024x1024']
export default function ImageGen() {
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
const { t } = useTranslation('media')
// Opened from the Studio front page, the form starts from what it sent.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const [prompt, setPrompt] = useState(handoff.prompt)
const [negativePrompt, setNegativePrompt] = useState('')
const [size, setSize] = useState(SIZES.includes(handoff.size) ? handoff.size : '512x512')
const [count, setCount] = useState(Math.min(4, handoff.count || 1))
const [steps, setSteps] = useState('')
const [seed, setSeed] = useState('')
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
const [images, setImages] = useState([])
const [showAdvanced, setShowAdvanced] = useState(false)
const [sourceImage, setSourceImage] = useState(null)
const [sourceName, setSourceName] = useState('')
const [sourcePreview, setSourcePreview] = useState(null)
const [refImages, setRefImages] = useState([])
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('image')
const ws = useWorkspace({ type: 'images', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
// A variation starts from the picture it was made from, as the source image.
const wantsSource = handoff.edge === 'variation'
const source = useHandoffSource(handoff, wantsSource)
useEffect(() => {
if (source.status !== 'ready') return
blobToImageInput(source.blob).then((img) => { setSourceImage(img.base64); setSourcePreview(img.dataUrl); setSourceName(source.item?.title || '') }).catch(() => {})
}, [source])
const [lightboxIdx, setLightboxIdx] = useState(null)
// The body of the last request, kept so the panel can show what was actually
// sent rather than what the form currently holds.
const [lastRequest, setLastRequest] = useState(null)
// The images currently on screen (a picked history entry, else the latest run).
const displayImages = selectedEntry
? selectedEntry.results.map(r => ({ url: r.url, alt: selectedEntry.prompt }))
: images.map(img => ({ url: img.url || `data:image/png;base64,${img.b64_json}`, alt: prompt }))
const activeEntry = selectedEntry || (lastId ? historyProps.entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const run = async () => {
if (!prompt.trim()) { addToast(t('image.toasts.noPrompt'), 'warning'); return }
if (!model) { addToast(t('image.toasts.noModel'), 'warning'); return }
setLoading(true)
setImages([])
setError(null)
setLastId(null)
let combinedPrompt = prompt.trim()
if (negativePrompt.trim()) combinedPrompt += '|' + negativePrompt.trim()
const body = { model, prompt: combinedPrompt, n: count, size }
if (steps) body.step = parseInt(steps)
if (seed) body.seed = parseInt(seed)
if (sourceImage) body.file = sourceImage
if (refImages.length > 0) body.ref_images = refImages
setLastRequest(body)
try {
const data = await imageApi.generate(body)
const results = data?.data || []
setImages(results)
if (!results.length) {
addToast(t('image.toasts.noResults'), 'warning')
} else {
const urlResults = results.filter(r => r.url && !r.url.startsWith('data:')).map(r => ({ url: r.url }))
if (urlResults.length) {
setLastId(addEntry({ prompt: prompt.trim(), model, params: { size, count, steps, seed, negativePrompt: negativePrompt.trim() || undefined }, results: urlResults, parentId: handoff.from || undefined, edge: handoff.edge || undefined }))
}
selectEntry(null)
ws.end()
}
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
const handleRefImages = async (files) => {
const arr = []
for (const f of files) arr.push(await fileToBase64(f))
setRefImages(prev => [...prev, ...arr])
}
// Re-run with edits: this result's values go back into the form, and the form
// remembers them so what changes can be listed.
const rerun = (it) => {
const entry = historyProps.entries.find(e => e.id === it.id)
if (!entry) return
const p = entry.params || {}
const next = {
prompt: entry.prompt || '',
model: entry.model || model,
size: SIZES.includes(p.size) ? p.size : size,
count: String(Math.min(4, Number(p.count) || 1)),
steps: p.steps ? String(p.steps) : '',
seed: p.seed ? String(p.seed) : '',
negative: p.negativePrompt || '',
}
setPrompt(next.prompt); setModel(next.model); setSize(next.size); setCount(Number(next.count))
setSteps(next.steps); setSeed(next.seed); setNegativePrompt(next.negative)
if (next.steps || next.seed || next.negative) setShowAdvanced(true)
selectEntry(null)
ws.begin(next)
}
const labels = {
prompt: t('studio.workspace.fields.prompt'), model: t('studio.workspace.fields.model'), size: t('studio.workspace.fields.size'),
count: t('studio.workspace.fields.count'), steps: t('studio.workspace.fields.steps'), seed: t('studio.workspace.fields.seed'),
negative: t('studio.workspace.fields.negative'),
}
const current = { prompt, model, size, count: String(count), steps, seed, negative: negativePrompt }
const changes = ws.changes(current, labels)
const changed = new Set(changes.map(c => c.field))
const installed = ws.installed.byType.images
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.images') })
: !prompt.trim() ? t('studio.composer.whyPrompt') : ''
const dims = parseSize(size)
return (
<Workspace type="images">
<ComposeCard
ws={ws}
icon="image"
title={t('image.title')}
lede={t('studio.workspace.lede.images')}
onSubmit={(e) => { e.preventDefault(); run() }}
handoff={<HandoffNote source={source} handoff={handoff} wanted={wantsSource} onClear={() => { setSourceImage(null); setSourceName('') }} />}
sources={<>
<SourceChip
label={t('studio.workspace.sources.startImage')}
accept="image/*"
name={sourceImage ? (sourceName || t('studio.workspace.sources.chosen')) : ''}
preview={sourceImage ? sourcePreview : undefined}
onFiles={async (file) => { const img = await blobToImageInput(file); setSourceImage(img.base64); setSourcePreview(img.dataUrl); setSourceName(file.name) }}
onClear={() => { setSourceImage(null); setSourceName('') }}
/>
<SourceChip
label={t('image.labels.refImages')}
accept="image/*"
multiple
count={refImages.length}
name={refImages.length ? t('image.labels.refImagesAdded', { count: refImages.length }) : ''}
onFiles={handleRefImages}
onClear={() => setRefImages([])}
/>
</>}
model={model}
noModel={noModel ? { type: 'images', label: t('studio.tabs.images'), onChanged: ws.installed.refetch } : null}
options={<>
<ModelChip value={model} onChange={setModel} capability={CAP_IMAGE} changed={changed.has('model')} />
<ChipSelect label={t('image.labels.size')} value={size} onChange={setSize} options={SIZES.map(s => ({ value: s, label: s.replace('x', ' × ') }))} changed={changed.has('size')} testId="ws-size" />
<ChipSelect label={t('image.labels.count')} value={String(count)} onChange={(v) => setCount(parseInt(v) || 1)} options={[1, 2, 3, 4].map(n => ({ value: String(n), label: t('studio.composer.countOption', { count: n }) }))} mono={false} changed={changed.has('count')} testId="ws-count" />
</>}
fold={
<Fold
label={t('image.labels.advanced')}
summary={foldSummary([steps && `${labels.steps} ${steps}`, seed && `${labels.seed} ${seed}`, negativePrompt.trim() && t('image.labels.negativePrompt').toLowerCase()].filter(Boolean), [t('image.labels.negativePrompt').toLowerCase(), labels.steps, labels.seed])}
open={showAdvanced}
onToggle={() => setShowAdvanced(v => !v)}
id="image-advanced-options"
>
<Field label={t('image.labels.negativePrompt')} htmlFor="image-negative" changed={changed.has('negative')}>
<textarea id="image-negative" className="textarea" value={negativePrompt} onChange={(e) => setNegativePrompt(e.target.value)} placeholder={t('image.labels.negativePromptPlaceholder')} rows={2} />
</Field>
<div className="ws-grid">
<Field label={t('image.labels.steps')} htmlFor="image-steps" changed={changed.has('steps')}>
<input id="image-steps" className="dk-input" type="number" value={steps} onChange={(e) => setSteps(e.target.value)} placeholder={t('image.labels.stepsPlaceholder')} />
</Field>
<Field label={t('image.labels.seed')} htmlFor="image-seed" changed={changed.has('seed')}>
<input id="image-seed" className="dk-input" type="number" value={seed} onChange={(e) => setSeed(e.target.value)} placeholder={t('image.labels.seedPlaceholder')} />
</Field>
</div>
</Fold>
}
changes={changes}
submit={{ label: t('image.actions.generate'), busyLabel: t('image.actions.generating'), busy: loading, disabled: !model || !prompt.trim() || noModel, why }}
>
<textarea
className="ws-prompt textarea"
value={prompt}
onChange={(e) => setPrompt(e.target.value)}
placeholder={t('image.labels.promptPlaceholder')}
aria-label={t('image.labels.prompt')}
rows={3}
data-changed={changed.has('prompt') || undefined}
onKeyDown={(e) => { if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); run() } }}
/>
{!prompt && <Starters type="images" onPick={setPrompt} />}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('image.actions.generating')} detail={[model, size, t('studio.composer.countOption', { count })].join(' · ')} tiles={count} ratio={dims ? dims.width / dims.height : 1} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : displayImages.length > 0 ? (
<ResultCard
ws={ws}
item={item}
title={item?.title || prompt}
meta={[item?.model || model, item?.params?.size || size, item?.params?.seed ? `${labels.seed} ${item.params.seed}` : '']}
download={displayImages.length === 1 ? { href: displayImages[0].url, name: `image-${Date.now()}.png` } : null}
onRerun={rerun}
>
<div className={`ws-grid-images${displayImages.length === 1 ? ' ws-grid-images--one' : ''} media-result-grid`}>
{displayImages.map((im, i) => (
<button type="button" key={i} className="media-result-thumb" onClick={() => setLightboxIdx(i)} title={t('image.actions.view')} aria-label={t('image.actions.view')}>
<img src={im.url} alt={im.alt} />
<span className="media-result-thumb__zoom" aria-hidden="true"><Icon name="maximize" /></span>
</button>
))}
</div>
</ResultCard>
) : (
<EmptyRun icon="image" text={t('image.empty')} />
)}
<RequestPanel endpoint="/v1/images/generations" body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={activeEntry?.id}
onSelect={historyProps.onSelect}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
{lightboxIdx !== null && (
<Lightbox images={displayImages} index={lightboxIdx} onIndex={setLightboxIdx} onClose={() => setLightboxIdx(null)} />
)}
</Workspace>
)
}
addons/studio/ui/src/pages/Sound.jsx
+294
View File
@@ -0,0 +1,294 @@
import { useState } from 'react'
// eslint-disable-next-line no-unused-vars
import HomeModelPicker from '../components/home/HomeModelPicker'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
import { useParams, useOutletContext } from 'react-router-dom'
import { CAP_SOUND_GENERATION } from '../utils/capabilities'
// eslint-disable-next-line no-unused-vars
import WaveformPlayer from '../components/audio/WaveformPlayer'
import { soundApi } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff } from '../hooks/useStudioHandoff'
import { useTranslation } from 'react-i18next'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ChipInput, ToggleChip, ModelChip, Field, Fold, Starters, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
import { BACKEND_DEFAULT, useSoundExpansionModel } from '../hooks/useSoundExpansionModel'
export default function Sound() {
const { t } = useTranslation('media')
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
// Opened from the Studio front page, the form starts from what it sent.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const [mode, setMode] = useState('simple')
const expansion = useSoundExpansionModel()
const [text, setText] = useState(handoff.prompt)
const [instrumental, setInstrumental] = useState(false)
const [vocalLanguage, setVocalLanguage] = useState('')
const [caption, setCaption] = useState('')
const [lyrics, setLyrics] = useState('')
const [think, setThink] = useState(false)
const [bpm, setBpm] = useState('')
const [duration, setDuration] = useState('')
const [keyscale, setKeyscale] = useState('')
const [language, setLanguage] = useState('')
const [timesignature, setTimesignature] = useState('')
const [showMore, setShowMore] = useState(false)
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
// What was actually sent, so the panel records rather than predicts.
const [lastRequest, setLastRequest] = useState(null)
const [audioUrl, setAudioUrl] = useState(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('sound')
const ws = useWorkspace({ type: 'sound', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
const activeEntry = selectedEntry || (lastId ? historyProps.entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const run = async () => {
if (!model) { addToast(t('sound.toasts.noModel'), 'warning'); return }
const body = { model_id: model }
if (mode === 'simple') {
if (!text.trim()) { addToast(t('sound.toasts.noPrompt'), 'warning'); return }
if (!expansion.value) { addToast(t('sound.expansion.choose'), 'warning'); return }
if (expansion.value !== BACKEND_DEFAULT) body.expansion_model = expansion.value
body.text = text.trim()
body.instrumental = instrumental
if (vocalLanguage.trim()) body.vocal_language = vocalLanguage.trim()
} else {
if (!caption.trim() && !lyrics.trim()) { addToast(t('studio.workspace.sound.noCaption'), 'warning'); return }
if (caption.trim()) body.caption = caption.trim()
if (lyrics.trim()) body.lyrics = lyrics.trim()
body.think = think
if (bpm) body.bpm = parseInt(bpm)
if (duration) body.duration_seconds = parseFloat(duration)
if (keyscale.trim()) body.keyscale = keyscale.trim()
if (language.trim()) body.language = language.trim()
if (timesignature.trim()) body.timesignature = timesignature.trim()
}
setLoading(true)
setAudioUrl(null)
setError(null)
setLastId(null)
setLastRequest(body)
try {
const { blob, serverUrl } = await soundApi.generate(body)
const url = URL.createObjectURL(blob)
setAudioUrl(url)
addToast(t('studio.workspace.sound.generated'), 'success')
const promptText = mode === 'simple' ? text.trim() : (caption.trim() || lyrics.trim())
if (serverUrl) {
// Every field of the form is kept, so Re-run with edits can put it back.
const params = mode === 'simple'
? { mode, expansionModel: expansion.value, instrumental, vocalLanguage: vocalLanguage.trim() || undefined }
: { mode, caption: caption.trim() || undefined, lyrics: lyrics.trim() || undefined, think, bpm, duration, keyscale, language, timesignature }
setLastId(addEntry({ prompt: promptText, model, params, results: [{ url: serverUrl }], parentId: handoff.from || undefined, edge: handoff.edge || undefined }))
}
selectEntry(null)
ws.end()
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
const rerun = (it) => {
const entry = historyProps.entries.find(e => e.id === it.id)
if (!entry) return
const p = entry.params || {}
const advanced = p.mode === 'advanced'
const next = {
model: entry.model || model, mode: advanced ? 'advanced' : 'simple',
expansionModel: advanced ? '' : (p.expansionModel || BACKEND_DEFAULT),
prompt: advanced ? (p.caption || '') : (entry.prompt || ''), lyrics: p.lyrics || '',
instrumental: p.instrumental ? 'on' : '', vocal: p.vocalLanguage || '', think: p.think ? 'on' : '',
bpm: p.bpm ? String(p.bpm) : '', duration: p.duration ? String(p.duration) : '', key: p.keyscale || '',
language: p.language || '', timesignature: p.timesignature || '',
}
setModel(next.model); setMode(next.mode); setInstrumental(!!next.instrumental); setVocalLanguage(next.vocal); setThink(!!next.think)
setBpm(next.bpm); setDuration(next.duration); setKeyscale(next.key); setLanguage(next.language); setTimesignature(next.timesignature)
if (advanced) { setCaption(next.prompt); setLyrics(next.lyrics) } else {
setText(next.prompt)
expansion.choose(next.expansionModel)
}
if (next.vocal || next.bpm || next.key || next.language || next.timesignature || next.think) setShowMore(true)
selectEntry(null)
ws.begin({ ...next, expansionModel: next.expansionModel === BACKEND_DEFAULT ? t('sound.expansion.backendDefault') : next.expansionModel })
}
const f = (key) => t(`studio.workspace.fields.${key}`)
const labels = {
expansionModel: t('sound.expansion.label'),
prompt: f('prompt'), model: f('model'), mode: f('mode'), lyrics: f('lyrics'), instrumental: f('instrumental'), vocal: f('language'),
think: f('think'), bpm: f('bpm'), duration: f('duration'), key: f('key'), language: f('language'), timesignature: f('timesignature'),
}
const current = {
expansionModel: mode === 'simple' ? (expansion.value === BACKEND_DEFAULT ? t('sound.expansion.backendDefault') : expansion.value) : '',
model, mode, prompt: mode === 'simple' ? text : caption, lyrics: mode === 'simple' ? '' : lyrics,
instrumental: mode === 'simple' && instrumental ? 'on' : '', vocal: mode === 'simple' ? vocalLanguage : '',
think: mode === 'advanced' && think ? 'on' : '', bpm: mode === 'advanced' ? bpm : '', duration: mode === 'advanced' ? duration : '',
key: mode === 'advanced' ? keyscale : '', language: mode === 'advanced' ? language : '', timesignature: mode === 'advanced' ? timesignature : '',
}
const changes = ws.changes(current, labels)
const changed = new Set(changes.map(c => c.field))
const installed = ws.installed.byType.sound
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const hasWords = mode === 'simple' ? !!text.trim() : !!(caption.trim() || lyrics.trim())
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.sound') })
: !hasWords ? t('studio.composer.whyPrompt')
: mode === 'simple' && !expansion.value ? t('sound.expansion.choose') : ''
const moreSet = mode === 'simple'
? [vocalLanguage && `${labels.vocal} ${vocalLanguage}`]
: [bpm && `${labels.bpm} ${bpm}`, keyscale && `${labels.key} ${keyscale}`, language && `${labels.language} ${language}`, timesignature && labels.timesignature, think && labels.think]
const moreAll = mode === 'simple' ? [labels.vocal] : [labels.bpm, labels.key, labels.language, labels.timesignature, labels.think]
return (
<Workspace type="sound">
<ComposeCard
ws={ws}
icon="music"
title={t('sound.title')}
lede={t('studio.workspace.lede.sound')}
onSubmit={(e) => { e.preventDefault(); run() }}
model={model}
noModel={noModel ? { type: 'sound', label: t('studio.tabs.sound'), onChanged: ws.installed.refetch } : null}
options={<>
<ModelChip value={model} onChange={setModel} capability={CAP_SOUND_GENERATION} changed={changed.has('model')} />
<div className="dk-segmented ws-mode" role="tablist" aria-label={t('studio.workspace.fields.mode')} data-changed={changed.has('mode') || undefined}>
<button type="button" role="tab" className="dk-seg" aria-selected={mode === 'simple'} onClick={() => setMode('simple')}>{t('sound.labels.simple')}</button>
<button type="button" role="tab" className="dk-seg" aria-selected={mode === 'advanced'} onClick={() => setMode('advanced')}>{t('sound.labels.advanced')}</button>
</div>
{mode === 'simple'
? <ToggleChip label={t('sound.labels.instrumental')} on={instrumental} onChange={setInstrumental} changed={changed.has('instrumental')} testId="ws-instrumental" />
: <ChipInput label={t('sound.labels.duration')} type="number" step="0.1" value={duration} onChange={setDuration} placeholder={t('studio.workspace.auto')} changed={changed.has('duration')} testId="ws-duration" />}
</>}
fold={
<Fold
label={t('studio.workspace.sound.more')}
summary={foldSummary(moreSet.filter(Boolean), moreAll)}
open={showMore}
onToggle={() => setShowMore(v => !v)}
id="sound-more-options"
>
{mode === 'simple' ? (
<Field label={t('sound.labels.vocalLanguage')} htmlFor="sound-vocal" changed={changed.has('vocal')}>
<input id="sound-vocal" className="dk-input" value={vocalLanguage} onChange={(e) => setVocalLanguage(e.target.value)} placeholder={t('sound.labels.vocalLanguagePlaceholder')} />
</Field>
) : (
<>
<div className="ws-grid">
<Field label={t('sound.labels.bpm')} htmlFor="sound-bpm" changed={changed.has('bpm')}><input id="sound-bpm" className="dk-input" type="number" value={bpm} onChange={(e) => setBpm(e.target.value)} /></Field>
<Field label={t('sound.labels.keyscale')} htmlFor="sound-key" changed={changed.has('key')}><input id="sound-key" className="dk-input" value={keyscale} onChange={(e) => setKeyscale(e.target.value)} /></Field>
<Field label={t('sound.labels.language')} htmlFor="sound-language" changed={changed.has('language')}><input id="sound-language" className="dk-input" value={language} onChange={(e) => setLanguage(e.target.value)} /></Field>
<Field label={t('sound.labels.timesignature')} htmlFor="sound-timesig" changed={changed.has('timesignature')}><input id="sound-timesig" className="dk-input" value={timesignature} onChange={(e) => setTimesignature(e.target.value)} /></Field>
</div>
<label className="ws-check" data-changed={changed.has('think') || undefined}>
<input type="checkbox" checked={think} onChange={(e) => setThink(e.target.checked)} />
<span>{t('sound.labels.thinkMode')}</span>
</label>
</>
)}
</Fold>
}
changes={changes}
submit={{ label: t('sound.actions.generate'), busyLabel: t('sound.actions.generating'), busy: loading, disabled: !model || !hasWords || noModel || (mode === 'simple' && !expansion.value), why, icon: 'music' }}
>
{mode === 'simple' ? (
<>
<textarea
className="ws-prompt textarea"
value={text}
onChange={(e) => setText(e.target.value)}
placeholder={t('sound.labels.promptPlaceholder')}
aria-label={t('sound.labels.prompt')}
rows={3}
data-changed={changed.has('prompt') || undefined}
/>
{!text && <Starters type="sound" onPick={setText} />}
<Field label={t('sound.expansion.label')} htmlFor="sound-expansion-model" changed={changed.has('expansionModel')}>
<HomeModelPicker
id="sound-expansion-model"
describedBy="sound-expansion-help"
labels={{ label: t('sound.expansion.label'), heading: t('sound.expansion.label') }}
placeholder={t('sound.expansion.placeholder')}
value={expansion.value === BACKEND_DEFAULT ? '' : expansion.value}
onChange={expansion.choose}
models={expansion.models}
loading={expansion.loading}
disabled={expansion.loading}
autoSelect={false}
showWarm={false}
rememberSelection={false}
defaultChoice={t('sound.expansion.backendDefault')}
defaultSelected={expansion.value === BACKEND_DEFAULT}
onChooseDefault={() => expansion.choose(BACKEND_DEFAULT)}
/>
<p id="sound-expansion-help" className="text-note" role="status">{expansion.loading
? t('sound.expansion.loading')
: expansion.error ? t('sound.expansion.error')
: expansion.models.length === 0 ? t('sound.expansion.noChat')
: t('sound.expansion.help')}</p>
</Field>
</>
) : (
<>
<Field label={t('sound.labels.caption')} htmlFor="sound-caption" changed={changed.has('prompt')}>
<textarea id="sound-caption" className="ws-prompt textarea" value={caption} onChange={(e) => setCaption(e.target.value)} rows={2} />
</Field>
<Field label={t('sound.labels.lyrics')} htmlFor="sound-lyrics" changed={changed.has('lyrics')}>
<textarea id="sound-lyrics" className="ws-prompt textarea" value={lyrics} onChange={(e) => setLyrics(e.target.value)} placeholder={t('sound.labels.lyricsPlaceholder')} rows={3} />
</Field>
</>
)}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('sound.actions.generating')} detail={[model, mode === 'advanced' && duration ? `${duration} s` : ''].filter(Boolean).join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : (selectedEntry || audioUrl) ? (
<ResultCard
ws={ws}
item={item}
title={item?.title || (mode === 'simple' ? text : (caption || lyrics))}
meta={[item?.model || model]}
download={{ href: selectedEntry ? selectedEntry.results[0]?.url : audioUrl, name: `sound-${new Date().toISOString().slice(0, 10)}.wav` }}
onRerun={rerun}
>
<div className="ws-audio">
<WaveformPlayer src={selectedEntry ? selectedEntry.results[0]?.url : audioUrl} height={96} />
{selectedEntry && <p className="ws-quote">&ldquo;{selectedEntry.prompt}&rdquo;</p>}
</div>
</ResultCard>
) : (
<EmptyRun icon="music" text={t('sound.empty')} />
)}
<RequestPanel endpoint="/sound-generation" body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={activeEntry?.id}
onSelect={historyProps.onSelect}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
</Workspace>
)
}
addons/studio/ui/src/pages/Studio.jsx
+144
View File
@@ -0,0 +1,144 @@
import { useEffect, useMemo } from 'react'
import { useNavigate, useParams, useSearchParams } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import ImageGen from './ImageGen'
import VideoGen from './VideoGen'
import ThreeDGen from './ThreeDGen'
import TTS from './TTS'
import Sound from './Sound'
import AudioTransform from './AudioTransform'
import Diarization from './Diarization'
import StudioOverview from './StudioOverview'
import { useAuth } from '../context/AuthContext'
import { useModels } from '../hooks/useModels'
import {
CAP_DIARIZATION, CAP_IMAGE, CAP_VIDEO, CAP_3D, CAP_3D_ANIMATION, CAP_TTS, CAP_SOUND_GENERATION, CAP_AUDIO_TRANSFORM,
} from '../utils/capabilities'
import { upscaleModels } from '../utils/upscale'
import Icon from '../components/Icon'
import '../components/studio/studio.css'
// One table for the six generators: the capability that makes a modality
// usable, the feature flag that can remove it entirely, and the group it reads
// under. Studio owns this so the tab strip and the overview cannot disagree
// about what exists.
const MODALITIES = [
{ key: 'diarization', capability: CAP_DIARIZATION, icon: 'users', group: 'voice', feature: 'audio_diarization' },
{ key: 'images', capability: CAP_IMAGE, icon: 'image', group: 'create' },
{ key: 'video', capability: CAP_VIDEO, icon: 'video', group: 'create' },
{ key: 'threed', capability: CAP_3D, icon: 'cube', group: 'create', feature: '3d' },
{ key: 'tts', capability: CAP_TTS, icon: 'headphones', group: 'voice' },
{ key: 'sound', capability: CAP_SOUND_GENERATION, icon: 'music', group: 'voice' },
{ key: 'transform', capability: CAP_AUDIO_TRANSFORM, icon: 'waveform', group: 'transform', feature: 'audio_transform' },
]
const OVERVIEW_TAB = { key: 'overview', icon: 'compass' }
const TAB_COMPONENTS = {
diarization: Diarization,
images: ImageGen,
video: VideoGen,
threed: ThreeDGen,
tts: TTS,
sound: Sound,
transform: AudioTransform,
}
export default function Studio() {
const { t } = useTranslation('media')
const { hasFeature } = useAuth()
const navigate = useNavigate()
const { tab: pathTab } = useParams()
const [searchParams] = useSearchParams()
// Once, unfiltered. useModels(capability) fetches the whole list and filters
// in the browser, so a hook per modality would be six identical requests to
// /api/models/capabilities on every mount.
const { models, loading: modelsLoading, error: modelsError, refetch: refetchModels } = useModels()
// A modality whose feature is off is not listed at all. That is a different
// thing from having no model, and the two must not look alike.
const available = useMemo(
() => MODALITIES.filter(m => !m.feature || hasFeature(m.feature)),
[hasFeature],
)
const modalities = useMemo(() => available.map(m => ({
...m,
installed: models
.filter(model => model.capabilities?.includes(m.capability) ||
(m.key === 'threed' && model.capabilities?.includes(CAP_3D_ANIMATION)))
.map(model => model.id),
})), [available, models])
const tabs = [OVERVIEW_TAB, ...available]
// Anything still arriving with ?tab= is sent to the path form once, replacing
// the history entry so Back does not bounce between the two spellings.
const legacyTab = searchParams.get('tab')
useEffect(() => {
if (legacyTab) navigate(`/app/studio/${legacyTab}`, { replace: true })
}, [legacyTab, navigate])
// Overview is the fallback for anything unrecognised or gated off. Landing on
// Images was never a decision, only the first entry in an array.
const activeTab = tabs.some(tab => tab.key === pathTab) ? pathTab : 'overview'
const setTab = (key) => navigate(key === 'overview' ? '/app/studio' : `/app/studio/${key}`)
const dotFor = (tab) => {
if (tab.key === 'overview') return null
const known = modalities.find(m => m.key === tab.key)
return known?.installed.length > 0 ? 'on' : 'off'
}
// On a phone the tab row scrolls sideways; keep the current type in view.
useEffect(() => {
document.querySelector('.studio-tab-active')?.scrollIntoView?.({ inline: 'center', block: 'nearest' })
}, [activeTab])
const ActiveComponent = TAB_COMPONENTS[activeTab]
return (
<div>
<div className="studio-tabs">
{tabs.map(tab => {
const dot = dotFor(tab)
return (
<button
key={tab.key}
data-tab={tab.key}
className={`studio-tab${activeTab === tab.key ? ' studio-tab-active' : ''}`}
onClick={() => setTab(tab.key)}
>
<Icon name={tab.icon} />
<span>{tab.key === 'overview' ? t('studio.tabs.overview') : t(`studio.tabs.${tab.key}`)}</span>
{/* A filled dot means a model on this machine serves the
modality, a hollow one means none does. The dot is drawn
quietly in text colours; the same fact is spoken for
readers who cannot see it. */}
{dot && (
<>
<span className={`studio-tab__dot studio-tab__dot--${dot}`} aria-hidden="true" />
<span className="sr-only">{t(`studio.tabState.${dot}`)}</span>
</>
)}
</button>
)
})}
</div>
{ActiveComponent ? (
<ActiveComponent />
) : (
<StudioOverview
modalities={modalities}
upscalers={upscaleModels(models)}
modelsLoading={modelsLoading}
modelsError={modelsError}
refetchModels={refetchModels}
/>
)}
</div>
)
}
addons/studio/ui/src/pages/StudioOverview.jsx
+129
View File
@@ -0,0 +1,129 @@
import { useCallback, useState } from 'react'
import { useNavigate, useSearchParams } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import PageHeader from '../components/PageHeader'
import Icon from '../components/Icon'
// eslint-disable-next-line no-unused-vars
import StudioComposer from '../components/studio/StudioComposer'
// eslint-disable-next-line no-unused-vars
import WorkMasonry from '../components/studio/WorkMasonry'
// eslint-disable-next-line no-unused-vars
import LineageView from '../components/studio/LineageView'
import { useStudioWork } from '../hooks/useStudioWork'
import { readLastModel } from '../utils/lastModel'
import { TYPE_ORDER } from '../utils/studioWork'
import '../components/studio/studio.css'
// The Studio front page: a prompt box that suggests what to make, and under it
// the things already made, with results that came from each other stacked into
// projects. A result or a stack opens as a lineage board at ?work=<id>, so Back
// and a reload both land where you were.
//
// The page does not generate anything itself. The composer opens the right
// workspace with the prompt filled in; each workspace is still where a run
// happens, is recorded, and shows its request.
//
// modalities the workspaces this person can use, each with the ids of the
// models installed for it
// modelsLoading true until the first answer about installed models
// modelsError set when that answer could not be read
// refetchModels ask again, used while a model installs
export default function StudioOverview({ modalities, upscalers, modelsLoading, modelsError, refetchModels }) {
const { t } = useTranslation('media')
const navigate = useNavigate()
const [searchParams, setSearchParams] = useSearchParams()
const { items, ready, addImageEntry, toggleFavourite, clearHistory } = useStudioWork()
const [filter, setFilter] = useState('all')
const [group, setGroup] = useState(true)
// The composer's state lives here, so opening a lineage and coming back, or a
// model installing, never costs the words that were typed.
const [draft, setDraft] = useState({ type: '', text: '', models: {}, sizes: {}, count: 1, sourceId: '' })
const workId = searchParams.get('work')
const types = TYPE_ORDER.filter(k => modalities.some(m => m.key === k))
// The model a type opens with: the one last picked in its workspace when it is
// still installed, else the first installed.
const defaultModel = useCallback((type, ids) => {
const capability = modalities.find(m => m.key === type)?.capability
const last = readLastModel(capability)
return ids.includes(last) ? last : (ids[0] || '')
}, [modalities])
const open = (id) => setSearchParams({ work: id })
const close = useCallback(() => setSearchParams({}), [setSearchParams])
const handoff = useCallback((path) => navigate(path), [navigate])
if (workId) {
return (
<div data-testid="studio-overview" className="page-pad studio-front">
{!ready ? (
<div className="dk-skeleton studio-skel studio-skel--board" aria-busy="true" />
) : (
<LineageView
key={workId}
items={items}
upscalers={upscalers}
onAddImage={addImageEntry}
workId={workId}
modalities={modalities}
defaultModel={defaultModel}
onClose={close}
onToggleFavourite={toggleFavourite}
onHandoff={handoff}
onModelsChanged={refetchModels}
/>
)}
</div>
)
}
return (
<div data-testid="studio-overview" className="page-pad studio-front">
<PageHeader
title={t('studio.overview.title')}
supporting={t('studio.overview.subtitle')}
actions={<span className="studio-key-hint"><kbd className="dk-kbd">/</kbd> {t('studio.composer.startTyping')}</span>}
/>
{modelsError ? (
<div className="studio-note studio-note--error studio-note--standalone" role="alert" data-testid="studio-models-error">
<h3>{t('studio.error.title')}</h3>
<p>{t('studio.error.body')}</p>
<div className="studio-note__row">
<button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={refetchModels}>
<Icon name="refresh" /> {t('studio.error.retry')}
</button>
</div>
</div>
) : modelsLoading ? (
<section className="studio-composer" aria-busy="true" data-testid="studio-composer-loading">
<div className="dk-skeleton studio-skel studio-skel--composer" />
</section>
) : (
<StudioComposer
draft={draft}
setDraft={setDraft}
modalities={modalities}
items={items}
onHandoff={handoff}
onModelsChanged={refetchModels}
defaultModel={defaultModel}
/>
)}
<WorkMasonry
items={items}
ready={ready}
types={types}
filter={filter}
onFilter={setFilter}
group={group}
onGroup={setGroup}
onOpen={open}
onToggleFavourite={toggleFavourite}
onClear={clearHistory}
/>
</div>
)
}
addons/studio/ui/src/pages/TTS.jsx
+266
View File
@@ -0,0 +1,266 @@
import { useEffect, useMemo, useRef, useState } from 'react'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
// eslint-disable-next-line no-unused-vars
import { Link, useParams, useOutletContext, useSearchParams } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import { CAP_TTS } from '../utils/capabilities'
// eslint-disable-next-line no-unused-vars
import WaveformPlayer from '../components/audio/WaveformPlayer'
import { ttsApi } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff } from '../hooks/useStudioHandoff'
import { useModels } from '../hooks/useModels'
import { useVoiceProfiles } from '../hooks/useVoiceProfiles'
import { useAuth } from '../context/AuthContext'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ChipSelect, ChipInput, ModelChip, Field, Fold, Starters, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
import Icon from '../components/Icon'
function formatProfileDuration(milliseconds) {
const seconds = Math.round((milliseconds || 0) / 1000)
return seconds >= 60 ? `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, '0')}` : `${seconds}s`
}
export default function TTS() {
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
const { t } = useTranslation('media')
const { isAdmin } = useAuth()
const [searchParams] = useSearchParams()
const requestedVoiceID = searchParams.get('voice') || ''
// Opened from the Studio front page, the form starts from what it sent.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const [manualVoice, setManualVoice] = useState('')
const [voiceProfileID, setVoiceProfileID] = useState(requestedVoiceID)
const [text, setText] = useState(handoff.prompt)
const [instructions, setInstructions] = useState('')
const [showAdvanced, setShowAdvanced] = useState(false)
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
// What was actually sent, so the request panel records rather than predicts.
const [lastRequest, setLastRequest] = useState(null)
const [audioUrl, setAudioUrl] = useState(null)
const appliedVoiceLinkRef = useRef('')
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('tts')
const ws = useWorkspace({ type: 'tts', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
const { models } = useModels(CAP_TTS)
const selectedModel = useMemo(() => models.find(item => item.id === model), [models, model])
const compatibleModels = useMemo(() => models.filter(item => item.voice_cloning), [models])
const supportsVoiceProfiles = !!selectedModel?.voice_cloning
const { profiles, loading: profilesLoading, error: profilesError } = useVoiceProfiles({ enabled: supportsVoiceProfiles })
const selectedProfile = profiles.find(profile => profile.id === voiceProfileID) || null
// A library deep-link should land on a compatible model even when the last
// TTS model used was a named-speaker or non-cloning variant. Apply it once
// per requested profile so the user can still change models afterwards.
useEffect(() => {
if (!requestedVoiceID || models.length === 0 || appliedVoiceLinkRef.current === requestedVoiceID) return
const targetModel = selectedModel?.voice_cloning ? selectedModel : compatibleModels[0]
if (!targetModel) return
if (targetModel.id !== model) setModel(targetModel.id)
setVoiceProfileID(requestedVoiceID)
appliedVoiceLinkRef.current = requestedVoiceID
}, [requestedVoiceID, models, model, selectedModel, compatibleModels])
useEffect(() => {
if (selectedModel && !selectedModel.voice_cloning) setVoiceProfileID('')
}, [selectedModel])
const activeEntry = selectedEntry || (lastId ? historyProps.entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const run = async () => {
if (!text.trim()) { addToast(t('tts.toasts.noText'), 'warning'); return }
if (!model) { addToast(t('tts.toasts.noModel'), 'warning'); return }
setLoading(true)
setAudioUrl(null)
setError(null)
setLastId(null)
try {
const selectedVoice = supportsVoiceProfiles ? selectedProfile?.voice : manualVoice.trim()
const request = { model, input: text.trim() }
if (selectedVoice) request.voice = selectedVoice
if (instructions.trim()) request.instructions = instructions.trim()
setLastRequest(request)
const { blob, serverUrl } = await ttsApi.generate(request)
const url = URL.createObjectURL(blob)
setAudioUrl(url)
addToast(t('tts.toasts.generated'), 'success')
if (serverUrl) {
setLastId(addEntry({
prompt: text.trim(),
model,
params: {
...(selectedProfile
? { voice: selectedProfile.name, voiceId: selectedProfile.id }
: (!supportsVoiceProfiles && manualVoice.trim() ? { voice: manualVoice.trim() } : {})),
...(instructions.trim() ? { instructions: instructions.trim() } : {}),
},
results: [{ url: serverUrl }],
parentId: handoff.from || undefined,
edge: handoff.edge || undefined,
}))
}
selectEntry(null)
ws.end()
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
const rerun = (it) => {
const entry = historyProps.entries.find(e => e.id === it.id)
if (!entry) return
const p = entry.params || {}
const cloning = !!models.find(m => m.id === entry.model)?.voice_cloning
const byName = profiles.find(pr => pr.name === p.voice)
const next = {
prompt: entry.prompt || '', model: entry.model || model,
voice: cloning ? (p.voiceId || byName?.id || '') : (p.voice || ''),
instructions: p.instructions || '',
}
setText(next.prompt); setModel(next.model); setInstructions(next.instructions)
if (cloning) { setVoiceProfileID(next.voice); setManualVoice('') } else { setManualVoice(next.voice); setVoiceProfileID('') }
if (next.instructions) setShowAdvanced(true)
selectEntry(null)
ws.begin(next)
}
const f = (key) => t(`studio.workspace.fields.${key}`)
const labels = { prompt: f('prompt'), model: f('model'), voice: f('voice'), instructions: f('instructions') }
const current = { prompt: text, model, voice: supportsVoiceProfiles ? voiceProfileID : manualVoice, instructions }
const changes = ws.changes(current, labels)
const changed = new Set(changes.map(c => c.field))
const installed = ws.installed.byType.tts
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.tts') })
: !text.trim() ? t('studio.composer.whyPrompt') : ''
const shownSrc = selectedEntry ? selectedEntry.results[0]?.url : audioUrl
return (
<Workspace type="tts">
<ComposeCard
ws={ws}
icon="headphones"
title={t('tts.title')}
lede={t('studio.workspace.lede.tts')}
onSubmit={(e) => { e.preventDefault(); run() }}
model={model}
noModel={noModel ? { type: 'tts', label: t('studio.tabs.tts'), onChanged: ws.installed.refetch } : null}
options={<>
<ModelChip value={model} onChange={setModel} capability={CAP_TTS} changed={changed.has('model')} />
{supportsVoiceProfiles ? (
<ChipSelect
label={t('tts.labels.voice')}
value={voiceProfileID}
onChange={setVoiceProfileID}
disabled={profilesLoading}
mono={false}
changed={changed.has('voice')}
testId="ws-voice"
id="tts-voice"
options={[
{ value: '', label: profilesLoading ? t('tts.voiceLibrary.loading') : t('tts.voiceLibrary.modelDefault') },
...profiles.map(profile => ({ value: profile.id, label: `${profile.name}${profile.language ? ` · ${profile.language}` : ''}` })),
]}
/>
) : (
<ChipInput label={t('tts.labels.voice')} value={manualVoice} onChange={setManualVoice} placeholder={t('tts.labels.voicePlaceholder')} width="wide" changed={changed.has('voice')} testId="ws-voice" id="tts-voice" />
)}
</>}
fold={
<Fold
label={t('tts.labels.instructions')}
summary={foldSummary(instructions.trim() ? [instructions.trim().slice(0, 40)] : [], [t('tts.labels.instructionsHint')])}
open={showAdvanced}
onToggle={() => setShowAdvanced(v => !v)}
id="tts-advanced-options"
>
<Field label={t('tts.labels.instructions')} htmlFor="tts-instructions" changed={changed.has('instructions')} hint={t('tts.labels.instructionsHint')}>
<textarea id="tts-instructions" className="textarea" value={instructions} onChange={(e) => setInstructions(e.target.value)} placeholder={t('tts.labels.instructionsPlaceholder')} rows={3} />
</Field>
</Fold>
}
changes={changes}
submit={{ label: t('tts.actions.generate'), busyLabel: t('tts.actions.generating'), busy: loading, disabled: !model || !text.trim() || noModel, why, icon: 'headphones' }}
>
<textarea
className="ws-prompt textarea"
value={text}
onChange={(e) => setText(e.target.value)}
placeholder={t('tts.labels.inputPlaceholder')}
aria-label={t('tts.labels.input')}
rows={4}
data-changed={changed.has('prompt') || undefined}
/>
{!text && <Starters type="tts" onPick={setText} />}
{supportsVoiceProfiles && (
<div className="ws-voicebar" data-testid="ws-voicebar">
<span className="dk-badge dk-badge--ok">{t('tts.voiceLibrary.cloningReady')}</span>
{selectedProfile && (
<span className="ws-voicebar__who">
<span className="ws-avatar">{selectedProfile.name.slice(0, 2).toUpperCase()}</span>
<strong>{selectedProfile.name}</strong>
<small>{selectedProfile.language || t('voiceLibrary.metadata.languageUnknown')} · {formatProfileDuration(selectedProfile.audio?.duration_ms)}</small>
</span>
)}
{!profilesLoading && profiles.length === 0 && (
<span className="ws-hint">
{t('tts.voiceLibrary.empty')}{' '}
{isAdmin && <Link className="ws-link" to="/app/voice-library/new">{t('tts.voiceLibrary.create')}</Link>}
</span>
)}
{profilesError && <span className="ws-hint ws-hint--error" role="alert">{profilesError}</span>}
{isAdmin && profiles.length > 0 && <Link className="ws-link" to="/app/voice-library">{t('tts.voiceLibrary.manage')} <Icon name="arrow-right" /></Link>}
</div>
)}
{!supportsVoiceProfiles && <p className="ws-hint">{t('tts.voiceLibrary.namedVoiceHint')}</p>}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('tts.actions.generating')} detail={[model, current.voice && (selectedProfile?.name || manualVoice)].filter(Boolean).join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : shownSrc ? (
<ResultCard
ws={ws}
item={item}
title={item?.title || text}
meta={[item?.model || model, item?.params?.voice ? `${labels.voice} ${item.params.voice}` : '']}
download={{ href: shownSrc, name: `tts-${(item?.model || model)}-${new Date().toISOString().slice(0, 10)}.mp3` }}
onRerun={rerun}
>
<div className="ws-audio">
<WaveformPlayer src={shownSrc} height={96} audioTestId={selectedEntry ? 'history-audio' : undefined} />
<p className="ws-quote">&ldquo;{selectedEntry ? selectedEntry.prompt : text}&rdquo;</p>
</div>
</ResultCard>
) : (
<EmptyRun icon="headphones" text={t('tts.empty')} />
)}
<RequestPanel endpoint="/v1/audio/speech" body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={activeEntry?.id}
onSelect={historyProps.onSelect}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
</Workspace>
)
}
addons/studio/ui/src/pages/ThreeDGen.jsx
+368
View File
@@ -0,0 +1,368 @@
import { useEffect, useMemo, useState } from 'react'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
import { useParams, useOutletContext } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import { CAP_3D, CAP_3D_ANIMATION } from '../utils/capabilities'
import { useModels } from '../hooks/useModels'
// eslint-disable-next-line no-unused-vars
import AnimationOptions from '../components/AnimationOptions'
// eslint-disable-next-line no-unused-vars
import AnimationViewer from '../components/AnimationViewer'
// eslint-disable-next-line no-unused-vars
import LoadingSpinner from '../components/LoadingSpinner'
// eslint-disable-next-line no-unused-vars
import GlbViewer from '../components/GlbViewer'
// eslint-disable-next-line no-unused-vars
import MediaInput from '../components/biometrics/MediaInput'
import { threeDApi, fetchUrl } from '../utils/api'
import { use3DHistory } from '../hooks/use3DHistory'
import { useStudioHandoff, useHandoffSource, blobToImageInput } from '../hooks/useStudioHandoff'
// eslint-disable-next-line no-unused-vars
import HandoffNote from '../components/studio/HandoffNote'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ChipSelect, ModelChip, Field, Fold, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
import useObjectUrl from '../hooks/useObjectUrl'
import Icon from '../components/Icon'
const QUALITIES = ['auto', 'coarse', '512', '1024']
const BACKGROUNDS = ['auto', 'keep', 'black', 'white']
const MAX_3D_INPUT_BYTES = 32 * 1024 * 1024
const REMESH_DETAIL_COARSE = 2.5
const REMESH_DETAIL_FINE = 0.35
function remeshDetail(sliderValue) {
const position = Number(sliderValue) / 100
return REMESH_DETAIL_COARSE * Math.pow(REMESH_DETAIL_FINE / REMESH_DETAIL_COARSE, position)
}
function remeshedName(name = '3d-model.glb') {
return `${name.replace(/\.glb$/i, '')}-remeshed.glb`
}
// Small thumbnail of the conditioning image for the history list — full-size
// data URLs would bloat every IndexedDB entry for no visual gain.
async function makeThumb(dataUrl, size = 96) {
try {
const img = new Image()
await new Promise((resolve, reject) => {
img.onload = resolve
img.onerror = reject
img.src = dataUrl
})
const scale = size / Math.max(img.width, img.height, 1)
const canvas = document.createElement('canvas')
canvas.width = Math.max(1, Math.round(img.width * scale))
canvas.height = Math.max(1, Math.round(img.height * scale))
canvas.getContext('2d').drawImage(img, 0, 0, canvas.width, canvas.height)
return canvas.toDataURL('image/jpeg', 0.7)
} catch {
return null
}
}
export default function ThreeDGen() {
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
const { t } = useTranslation('media')
// Opened from the Studio front page, a picture it was made from becomes the
// conditioning image and the model it chose is selected.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const { models, loading: modelsLoading } = useModels()
const modelNames = useMemo(() => models.filter(item => item.capabilities?.some(cap => cap === CAP_3D || cap === CAP_3D_ANIMATION)).map(item => item.id), [models])
const selectedModel = models.find(item => item.id === model)
const animation = selectedModel?.three_d_operations?.find(operation => operation.id === 'animate')
const [animationInputs, setAnimationInputs] = useState({})
const [animationParams, setAnimationParams] = useState({})
const [requestEndpoint, setRequestEndpoint] = useState('/3d/generations')
const [image, setImage] = useState(null)
const [quality, setQuality] = useState('auto')
const [background, setBackground] = useState('auto')
const [steps, setSteps] = useState('')
const [textureSteps, setTextureSteps] = useState('')
const [guidance, setGuidance] = useState('')
const [seed, setSeed] = useState('')
const [showAdvanced, setShowAdvanced] = useState(false)
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
// What was actually sent, so the panel records rather than predicts.
const [lastRequest, setLastRequest] = useState(null)
const [result, setResult] = useState(null) // { blob, name, model }
const [lastId, setLastId] = useState(null)
const [remeshSlider, setRemeshSlider] = useState(82)
const [remeshState, setRemeshState] = useState(null) // { sourceBlob, blob?, name?, error? }
const [remeshLoading, setRemeshLoading] = useState(false)
const { entries, addEntry, deleteEntry, clearAll, selectEntry, selectedId, selectedEntry } = use3DHistory()
const ws = useWorkspace({ type: 'threed', entries })
const wantsSource = handoff.edge === 'to-3d'
const handoffSource = useHandoffSource(handoff, wantsSource)
useEffect(() => {
if (handoffSource.status !== 'ready') return
blobToImageInput(handoffSource.blob).then(setImage).catch(() => {})
}, [handoffSource])
const source = selectedEntry
? { ...selectedEntry, blob: selectedEntry.glb }
: result
const showingRemesh = !!source && remeshState?.sourceBlob === source.blob && !!remeshState.blob
const active = showingRemesh ? { blob: remeshState.blob, name: remeshState.name } : source
const remeshError = source && remeshState?.sourceBlob === source.blob ? remeshState.error : null
const detail = remeshDetail(remeshSlider)
const downloadUrl = useObjectUrl(active?.blob)
const sourceModel = models.find(item => item.id === source?.model)
const canRemesh = source?.outputType !== 'skeleton_animation' && (sourceModel?.three_d_operations?.some(operation => operation.id === 'remesh') ||
(!sourceModel?.three_d_operations && sourceModel?.capabilities?.includes(CAP_3D)))
const activeEntry = selectedEntry || (lastId ? entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const changeModel = (next) => {
setModel(next)
setAnimationInputs({})
setAnimationParams({})
setImage(null)
setQuality('auto')
setBackground('auto')
setSteps('')
setTextureSteps('')
setGuidance('')
setSeed('')
}
const restoreHistory = (id) => {
selectEntry(id)
const entry = entries.find(item => item.id === id)
if (entry?.operation === 'animate') {
setModel(entry.model)
setAnimationInputs(entry.inputs || {})
setAnimationParams(entry.params || {})
}
}
const run = async () => {
if (!animation && !image?.base64) { addToast(t('threed.toasts.noImage'), 'warning'); return }
if (!model) { addToast(t('threed.toasts.noModel'), 'warning'); return }
setLoading(true)
setResult(null)
setRemeshState(null)
setError(null)
setLastId(null)
const params = Object.fromEntries(Object.entries(animationParams).filter(([key, value]) => value !== '' && animation?.parameters.some(parameter => parameter.name === key)))
const body = animation
? { model, inputs: animationInputs, params, response_format: 'url' }
: { model, image: image.base64, quality, background, response_format: 'url' }
if (!animation) {
if (steps) body.step = parseInt(steps)
if (textureSteps) body.texture_steps = parseInt(textureSteps)
if (guidance) body.cfg_scale = parseFloat(guidance)
if (seed) body.seed = parseInt(seed)
}
// RequestPanel renders and copies its body. Keeping a multi-megabyte image
// there duplicates the upload in React and can starve the result render.
setLastRequest(animation ? body : { ...body, image: `<base64 ${image.mime || 'image'} omitted>` })
setRequestEndpoint(animation ? '/3d/animate' : '/3d/generations')
try {
const data = await (animation ? threeDApi.animate(body) : threeDApi.generate(body))
const url = data?.data?.[0]?.url
if (!url) {
addToast(t('threed.toasts.noResults'), 'warning')
return
}
const glbResp = await fetchUrl(url)
if (!glbResp.ok) throw new Error(`fetching the generated GLB failed: HTTP ${glbResp.status}`)
const glb = await glbResp.blob()
const name = url.split('/').pop()
const outputType = animation?.output || 'mesh'
setResult({ blob: glb, name, model, outputType })
selectEntry(null)
const inputThumb = !animation && image?.dataUrl ? await makeThumb(image.dataUrl) : null
const saved = await addEntry({
model,
params: animation ? params : { quality, background, steps, textureSteps, guidance, seed },
inputs: animation ? animationInputs : undefined,
operation: animation ? 'animate' : 'generate',
outputType,
inputThumb,
glb,
name,
parentId: handoff.from || undefined,
edge: handoff.edge || undefined,
label: handoffSource.item?.title || undefined,
})
setLastId(saved?.id || null)
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
const handleRemesh = async () => {
if (!source?.blob || !source.model) return
if (showingRemesh) {
setRemeshState(null)
return
}
const sourceBlob = source.blob
setRemeshLoading(true)
setRemeshState({ sourceBlob, error: null })
try {
const blob = await threeDApi.remesh(sourceBlob, source.model, detail)
setRemeshState({ sourceBlob, blob, name: remeshedName(source.name), error: null })
} catch (err) {
setRemeshState({ sourceBlob, error: err.message })
} finally {
setRemeshLoading(false)
}
}
const handleRemeshDetail = (value) => {
setRemeshSlider(value)
if (source && remeshState?.sourceBlob === source.blob) setRemeshState(null)
}
const installed = ws.installed.byType.threed
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.threed') })
: (!animation && !image?.base64) ? t('studio.workspace.threed.whyImage') : ''
const f = (key) => t(`studio.workspace.fields.${key}`)
const advancedSet = [steps && `${t('threed.labels.steps')} ${steps}`, textureSteps && `${t('threed.labels.textureSteps')} ${textureSteps}`, guidance && `${f('cfg')} ${guidance}`, seed && `${f('seed')} ${seed}`].filter(Boolean)
const qualityOptions = QUALITIES.map(q => ({ value: q, label: t(`threed.labels.quality_${q}`) }))
const backgroundOptions = BACKGROUNDS.map(b => ({ value: b, label: t(`threed.labels.background_${b}`) }))
return (
<Workspace type="threed">
<ComposeCard
ws={ws}
icon="cube"
title={t('threed.title')}
lede={t('studio.workspace.lede.threed')}
onSubmit={(e) => { e.preventDefault(); run() }}
handoff={<HandoffNote source={handoffSource} handoff={handoff} wanted={wantsSource} onClear={() => setImage(null)} />}
model={model}
noModel={noModel ? { type: 'threed', label: t('studio.tabs.threed'), onChanged: ws.installed.refetch } : null}
options={<>
<ModelChip value={model} onChange={changeModel} options={modelNames} loading={modelsLoading} disabled={loading} capability={CAP_3D} />
{!animation && <>
<ChipSelect label={t('threed.labels.quality')} value={quality} onChange={setQuality} options={qualityOptions} mono={false} testId="ws-quality" />
<ChipSelect label={t('threed.labels.background')} value={background} onChange={setBackground} options={backgroundOptions} mono={false} testId="ws-background" />
</>}
</>}
fold={animation ? null : (
<Fold
label={t('threed.labels.advanced')}
summary={foldSummary(advancedSet, [t('threed.labels.steps'), t('threed.labels.textureSteps'), t('threed.labels.guidance'), t('threed.labels.seed')])}
open={showAdvanced}
onToggle={() => setShowAdvanced(v => !v)}
id="threed-advanced-options"
>
<div className="ws-grid">
<Field label={t('threed.labels.steps')} htmlFor="threed-steps"><input id="threed-steps" className="dk-input" type="number" min="1" value={steps} onChange={(e) => setSteps(e.target.value)} placeholder="12" /></Field>
<Field label={t('threed.labels.textureSteps')} htmlFor="threed-texture-steps"><input id="threed-texture-steps" className="dk-input" type="number" min="1" value={textureSteps} onChange={(e) => setTextureSteps(e.target.value)} placeholder="12" /></Field>
<Field label={t('threed.labels.guidance')} htmlFor="threed-guidance"><input id="threed-guidance" className="dk-input" type="number" step="0.1" value={guidance} onChange={(e) => setGuidance(e.target.value)} placeholder="7.5" /></Field>
<Field label={t('threed.labels.seed')} htmlFor="threed-seed"><input id="threed-seed" className="dk-input" type="number" value={seed} onChange={(e) => setSeed(e.target.value)} placeholder={t('threed.labels.seedPlaceholder')} /></Field>
</div>
</Fold>
)}
submit={{ label: t('threed.actions.generate'), busyLabel: t('threed.actions.generating'), busy: loading, disabled: !model || noModel || (!animation && !image?.base64), why, icon: 'cube' }}
>
{animation ? (
<div className="ws-inputs ws-inputs--one">
<AnimationOptions key={model} operation={animation} inputs={animationInputs} onInputsChange={setAnimationInputs} params={animationParams} onParamsChange={setAnimationParams} />
</div>
) : (
<div className="ws-inputs ws-inputs--one">
<MediaInput
mode="image"
label={t('threed.labels.image')}
value={image}
onChange={setImage}
onError={(err) => addToast(err.message, 'error')}
maxBytes={MAX_3D_INPUT_BYTES}
idPrefix="threed"
/>
</div>
)}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('threed.actions.generating')} detail={[model, !animation && quality !== 'auto' ? quality : ''].filter(Boolean).join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : active?.blob ? (
<ResultCard
ws={ws}
item={item}
title={item?.title || active.name || t('studio.work.untitled.threed')}
meta={[source?.model, source?.params?.quality ? t(`threed.labels.quality_${source.params.quality}`, { defaultValue: source.params.quality }) : '']}
download={{ href: downloadUrl, name: active.name || `3d-${model || 'model'}.glb`, testId: 'glb-download' }}
onRerun={() => {}}
>
<div className="ws-viewer">
{active.outputType === 'skeleton_animation' ? <AnimationViewer blob={active.blob} /> : <GlbViewer blob={active.blob} />}
{canRemesh && <div className="threed-remesh-controls">
<div className="threed-remesh-heading">
<span>{t('threed.remesh.title')}</span>
<output htmlFor="threed-remesh-detail">{detail.toFixed(2)}%</output>
</div>
<input
id="threed-remesh-detail"
type="range"
min="0"
max="100"
step="1"
value={remeshSlider}
onChange={(e) => handleRemeshDetail(e.target.value)}
disabled={remeshLoading}
aria-label={t('threed.remesh.detail')}
/>
<div className="threed-remesh-scale" aria-hidden="true">
<span>{t('threed.remesh.coarser')}</span>
<span>{t('threed.remesh.finer')}</span>
</div>
<p className="form-hint">{t('threed.remesh.hint')}</p>
<button
type="button"
className="dk-btn dk-btn--secondary dk-btn--sm"
onClick={handleRemesh}
disabled={remeshLoading}
data-testid="glb-remesh"
>
{remeshLoading
? <><LoadingSpinner size="sm" /> {t('threed.actions.remeshing')}</>
: showingRemesh
? <><Icon name="undo" /> {t('threed.actions.showOriginal')}</>
: <><Icon name="boxes" /> {t('threed.actions.remesh')}</>}
</button>
{remeshError && <p className="form-error" role="alert">{remeshError}</p>}
{showingRemesh && <p className="threed-remesh-ready">{t('threed.remesh.ready')}</p>}
</div>}
</div>
</ResultCard>
) : (
<EmptyRun icon="cube" text={t('threed.empty')} />
)}
<RequestPanel endpoint={requestEndpoint} body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={selectedId}
activeId={activeEntry?.id}
onSelect={restoreHistory}
onDelete={deleteEntry}
onClear={clearAll}
/>
</Workspace>
)
}
addons/studio/ui/src/pages/VideoGen.jsx
+251
View File
@@ -0,0 +1,251 @@
import { useState, useEffect } from 'react'
// eslint-disable-next-line no-unused-vars
import RequestPanel from '../components/RequestPanel'
import { useParams, useOutletContext } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import { CAP_VIDEO } from '../utils/capabilities'
// eslint-disable-next-line no-unused-vars
import MediaInput from '../components/biometrics/MediaInput'
import { videoApi } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
import { useStudioHandoff, useHandoffSource, blobToImageInput } from '../hooks/useStudioHandoff'
// eslint-disable-next-line no-unused-vars
import HandoffNote from '../components/studio/HandoffNote'
import {
// eslint-disable-next-line no-unused-vars
Workspace, ComposeCard, ChipSelect, ChipInput, ModelChip, Field, Fold, SourceChip, PanelChip, Starters, RunArea, JobCard, FailedCard, ResultCard, ResultsStrip, EmptyRun,
} from '../components/studio/Workspace'
import { foldSummary, useWorkspace } from '../hooks/useWorkspace'
const SIZES = ['256x256', '512x512', '768x768', '1024x1024', '832x480', '1280x720']
export default function VideoGen() {
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
const { t } = useTranslation('media')
// Opened from the Studio front page, the form starts from what it sent.
const handoff = useStudioHandoff()
const [model, setModel] = useState(urlModel || handoff.model || '')
const [prompt, setPrompt] = useState(handoff.prompt)
const [negativePrompt, setNegativePrompt] = useState('')
const [size, setSize] = useState(SIZES.includes(handoff.size) ? handoff.size : '512x512')
const [seconds, setSeconds] = useState('')
const [fps, setFps] = useState('16')
const [frames, setFrames] = useState('')
const [steps, setSteps] = useState('')
const [seed, setSeed] = useState('')
const [cfgScale, setCfgScale] = useState('')
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
// What was actually sent, so the panel records rather than predicts.
const [lastRequest, setLastRequest] = useState(null)
const [videos, setVideos] = useState([])
const [showAdvanced, setShowAdvanced] = useState(false)
const [showAvatar, setShowAvatar] = useState(false)
const [startImage, setStartImage] = useState(null)
const [startName, setStartName] = useState('')
const [startPreview, setStartPreview] = useState(null)
const [endPreview, setEndPreview] = useState(null)
const [endImage, setEndImage] = useState(null)
const [endName, setEndName] = useState('')
const [audioInput, setAudioInput] = useState(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('video')
const ws = useWorkspace({ type: 'video', entries: historyProps.entries })
const [lastId, setLastId] = useState(null)
// Animating a picture starts from it as the start image.
const wantsSource = handoff.edge === 'animate'
const source = useHandoffSource(handoff, wantsSource)
useEffect(() => {
if (source.status !== 'ready') return
blobToImageInput(source.blob).then((img) => { setStartImage(img.base64); setStartPreview(img.dataUrl); setStartName(source.item?.title || '') }).catch(() => {})
}, [source])
const activeEntry = selectedEntry || (lastId ? historyProps.entries.find(e => e.id === lastId) : null) || null
const item = ws.itemById(activeEntry?.id)
const shown = selectedEntry
? selectedEntry.results.map(r => r.url)
: videos.map(v => v.url || `data:video/mp4;base64,${v.b64_json}`)
const run = async () => {
if (!prompt.trim()) { addToast(t('video.toasts.noPrompt'), 'warning'); return }
if (!model) { addToast(t('video.toasts.noModel'), 'warning'); return }
setLoading(true)
setVideos([])
setError(null)
setLastId(null)
const [w, h] = size.split('x').map(Number)
const body = { model, prompt: prompt.trim(), width: w, height: h, fps: parseInt(fps) || 16 }
if (negativePrompt.trim()) body.negative_prompt = negativePrompt.trim()
if (seconds) body.seconds = seconds
if (frames) body.num_frames = parseInt(frames)
if (steps) body.step = parseInt(steps)
if (seed) body.seed = parseInt(seed)
if (cfgScale) body.cfg_scale = parseFloat(cfgScale)
if (startImage) body.start_image = startImage
if (endImage) body.end_image = endImage
if (audioInput?.base64) body.audio = audioInput.base64
setLastRequest(body)
try {
const data = await videoApi.generate(body)
const results = data?.data || []
setVideos(results)
if (!results.length) {
addToast(t('video.toasts.noResults'), 'warning')
} else {
const urlResults = results.filter(r => r.url && !r.url.startsWith('data:')).map(r => ({ url: r.url }))
if (urlResults.length) {
setLastId(addEntry({ prompt: prompt.trim(), model, params: { size, fps, seconds, frames, steps, seed, cfgScale, negativePrompt: negativePrompt.trim() || undefined }, results: urlResults, parentId: handoff.from || undefined, edge: handoff.edge || undefined }))
}
selectEntry(null)
ws.end()
}
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
const rerun = (it) => {
const entry = historyProps.entries.find(e => e.id === it.id)
if (!entry) return
const p = entry.params || {}
const next = {
prompt: entry.prompt || '', model: entry.model || model, size: SIZES.includes(p.size) ? p.size : size,
seconds: p.seconds ? String(p.seconds) : '', fps: p.fps ? String(p.fps) : '16', frames: p.frames ? String(p.frames) : '',
steps: p.steps ? String(p.steps) : '', seed: p.seed ? String(p.seed) : '', cfg: p.cfgScale ? String(p.cfgScale) : '',
negative: p.negativePrompt || '',
}
setPrompt(next.prompt); setModel(next.model); setSize(next.size); setSeconds(next.seconds); setFps(next.fps)
setFrames(next.frames); setSteps(next.steps); setSeed(next.seed); setCfgScale(next.cfg); setNegativePrompt(next.negative)
if (next.frames || next.steps || next.seed || next.cfg || next.negative) setShowAdvanced(true)
selectEntry(null)
ws.begin(next)
}
const f = (key) => t(`studio.workspace.fields.${key}`)
const labels = {
prompt: f('prompt'), model: f('model'), size: f('size'), seconds: f('duration'), fps: f('fps'), frames: f('frames'),
steps: f('steps'), seed: f('seed'), cfg: f('cfg'), negative: f('negative'),
}
const current = { prompt, model, size, seconds, fps, frames, steps, seed, cfg: cfgScale, negative: negativePrompt }
const changes = ws.changes(current, labels)
const changed = new Set(changes.map(c => c.field))
const installed = ws.installed.byType.video
const noModel = !ws.installed.loading && !ws.installed.error && installed.length === 0
const why = noModel ? t('studio.composer.whyModel', { type: t('studio.tabs.video') })
: !prompt.trim() ? t('studio.composer.whyPrompt') : ''
return (
<Workspace type="video">
<ComposeCard
ws={ws}
icon="video"
title={t('video.title')}
lede={t('studio.workspace.lede.video')}
onSubmit={(e) => { e.preventDefault(); run() }}
handoff={<HandoffNote source={source} handoff={handoff} wanted={wantsSource} onClear={() => { setStartImage(null); setStartName('') }} />}
sources={<>
<SourceChip
label={t('video.labels.startImage')}
accept="image/*"
name={startImage ? (startName || t('studio.workspace.sources.chosen')) : ''}
preview={startImage ? startPreview : undefined}
onFiles={async (file) => { const img = await blobToImageInput(file); setStartImage(img.base64); setStartPreview(img.dataUrl); setStartName(file.name) }}
onClear={() => { setStartImage(null); setStartName('') }}
/>
<SourceChip
label={t('video.labels.endImage')}
accept="image/*"
name={endImage ? (endName || t('studio.workspace.sources.chosen')) : ''}
preview={endImage ? endPreview : undefined}
onFiles={async (file) => { const img = await blobToImageInput(file); setEndImage(img.base64); setEndPreview(img.dataUrl); setEndName(file.name) }}
onClear={() => { setEndImage(null); setEndName('') }}
/>
<PanelChip label={t('video.labels.avatarAudio')} open={showAvatar} onToggle={() => setShowAvatar(v => !v)} filled={!!audioInput?.base64} controls="video-avatar-panel" />
</>}
panels={showAvatar && (
<div id="video-avatar-panel" className="ws-panel">
<MediaInput mode="audio" label={t('video.labels.avatarAudio')} value={audioInput} onChange={setAudioInput} idPrefix="video-avatar" />
</div>
)}
model={model}
noModel={noModel ? { type: 'video', label: t('studio.tabs.video'), onChanged: ws.installed.refetch } : null}
options={<>
<ModelChip value={model} onChange={setModel} capability={CAP_VIDEO} changed={changed.has('model')} />
<ChipSelect label={t('video.labels.size')} value={size} onChange={setSize} options={SIZES.map(s => ({ value: s, label: s.replace('x', ' × ') }))} changed={changed.has('size')} testId="ws-size" />
<ChipInput label={t('video.labels.duration')} value={seconds} onChange={setSeconds} placeholder={t('studio.workspace.auto')} changed={changed.has('seconds')} testId="ws-duration" />
<ChipInput label={t('video.labels.fps')} type="number" value={fps} onChange={setFps} changed={changed.has('fps')} testId="ws-fps" />
</>}
fold={
<Fold
label={t('video.labels.advanced')}
summary={foldSummary([steps && `${labels.steps} ${steps}`, seed && `${labels.seed} ${seed}`, cfgScale && `${labels.cfg} ${cfgScale}`, frames && `${labels.frames} ${frames}`, negativePrompt.trim() && labels.negative].filter(Boolean), [labels.negative, labels.steps, labels.seed, labels.cfg, labels.frames])}
open={showAdvanced}
onToggle={() => setShowAdvanced(v => !v)}
id="video-advanced-options"
>
<Field label={t('image.labels.negativePrompt')} htmlFor="video-negative" changed={changed.has('negative')}>
<textarea id="video-negative" className="textarea" value={negativePrompt} onChange={(e) => setNegativePrompt(e.target.value)} rows={2} />
</Field>
<div className="ws-grid">
<Field label={t('image.labels.steps')} htmlFor="video-steps" changed={changed.has('steps')}><input id="video-steps" className="dk-input" type="number" value={steps} onChange={(e) => setSteps(e.target.value)} /></Field>
<Field label={t('video.labels.seed')} htmlFor="video-seed" changed={changed.has('seed')}><input id="video-seed" className="dk-input" type="number" value={seed} onChange={(e) => setSeed(e.target.value)} placeholder={t('video.labels.seedPlaceholder')} /></Field>
<Field label="CFG Scale" htmlFor="video-cfg" changed={changed.has('cfg')}><input id="video-cfg" className="dk-input" type="number" step="0.1" value={cfgScale} onChange={(e) => setCfgScale(e.target.value)} /></Field>
<Field label={t('video.labels.frames')} htmlFor="video-frames" changed={changed.has('frames')}><input id="video-frames" className="dk-input" type="number" min="1" value={frames} onChange={(e) => setFrames(e.target.value)} /></Field>
</div>
</Fold>
}
changes={changes}
submit={{ label: t('video.actions.generate'), busyLabel: t('video.actions.generating'), busy: loading, disabled: !model || !prompt.trim() || noModel, why, icon: 'video' }}
>
<textarea
className="ws-prompt textarea"
value={prompt}
onChange={(e) => setPrompt(e.target.value)}
placeholder={t('video.labels.promptPlaceholder')}
aria-label={t('video.labels.prompt')}
rows={3}
data-changed={changed.has('prompt') || undefined}
/>
{!prompt && <Starters type="video" onPick={setPrompt} />}
</ComposeCard>
<RunArea>
{loading ? (
<JobCard label={t('video.actions.generating')} detail={[model, size, `${fps} ${t('video.labels.fps').toLowerCase()}`].join(' · ')} />
) : error ? (
<FailedCard message={error} onRetry={run} />
) : shown.length > 0 ? (
<ResultCard
ws={ws}
item={item}
title={item?.title || prompt}
meta={[item?.model || model, item?.params?.size || size, item?.params?.fps ? `${item.params.fps} ${t('video.labels.fps').toLowerCase()}` : '', item?.params?.seed ? `${labels.seed} ${item.params.seed}` : '']}
download={shown.length === 1 ? { href: shown[0], name: `video-${Date.now()}.mp4` } : null}
onRerun={rerun}
>
{shown.map((src, i) => <video key={i} controls src={src} />)}
</ResultCard>
) : (
<EmptyRun icon="video" text={t('video.empty')} />
)}
<RequestPanel endpoint="/video" body={lastRequest} />
</RunArea>
<ResultsStrip
ws={ws}
selectedId={historyProps.selectedId}
activeId={activeEntry?.id}
onSelect={historyProps.onSelect}
onDelete={historyProps.onDelete}
onClear={historyProps.onClearAll}
/>
</Workspace>
)
}
addons/studio/ui/src/style.css
+1
View File
@@ -0,0 +1 @@
/* Styles of the Studio add-on. The page style sheets (components/studio/*.css) are added by the build. */
addons/studio/ui/src/utils/animationGlb.js
+128
View File
@@ -0,0 +1,128 @@
// SPDX-License-Identifier: MIT
// Skeleton GLBs contain animated nodes but no mesh. Keep their reader separate
// from the dense TRELLIS mesh path, whose buffer layouts and rendering differ.
export function parseAnimationGlb(buffer) {
const view = new DataView(buffer)
if (buffer.byteLength < 20 || view.getUint32(0, true) !== 0x46546c67 || view.getUint32(4, true) !== 2 || view.getUint32(8, true) !== buffer.byteLength) {
throw new Error('Invalid animation GLB')
}
let document, binary
for (let offset = 12; offset < buffer.byteLength;) {
if (offset + 8 > buffer.byteLength) throw new Error('Truncated GLB chunk')
const length = view.getUint32(offset, true)
const type = view.getUint32(offset + 4, true)
offset += 8
if (length % 4 || offset + length > buffer.byteLength) throw new Error('Invalid GLB chunk length')
if (type === 0x4e4f534a) document = JSON.parse(new TextDecoder().decode(new Uint8Array(buffer, offset, length)))
if (type === 0x004e4942) binary = buffer.slice(offset, offset + length)
offset += length
}
if (!document?.nodes?.length || document.nodes.length > 1024 || !binary) throw new Error('GLB has no supported skeleton')
const nodes = document.nodes.map(node => {
const translation = node.translation || [0, 0, 0]
const rotation = node.rotation || [0, 0, 0, 1]
if (!Array.isArray(translation) || translation.length !== 3 || !translation.every(Number.isFinite) ||
!Array.isArray(rotation) || rotation.length !== 4 || !rotation.every(Number.isFinite) || Math.hypot(...rotation) < 1e-8) {
throw new Error('Invalid skeleton transform')
}
if (node.matrix || (node.scale !== undefined && (!Array.isArray(node.scale) || node.scale.length !== 3 || node.scale.some(value => value !== 1)))) {
throw new Error('Unsupported skeleton transform')
}
const length = Math.hypot(...rotation)
return { name: node.name || '', parent: -1, translation, rotation: rotation.map(value => value / length) }
})
document.nodes.forEach((node, parent) => {
for (const child of node.children || []) {
if (!Number.isInteger(child) || !nodes[child] || nodes[child].parent !== -1 || child === parent) throw new Error('Invalid skeleton hierarchy')
nodes[child].parent = parent
}
})
const order = [], visiting = new Set(), visited = new Set()
function visit(index) {
if (visited.has(index)) return
if (visiting.has(index)) throw new Error('Cyclic skeleton hierarchy')
visiting.add(index)
if (nodes[index].parent >= 0) visit(nodes[index].parent)
visiting.delete(index)
visited.add(index)
order.push(index)
}
nodes.forEach((_, index) => visit(index))
function readAccessor(index, type, components) {
const accessor = document.accessors?.[index]
const bytes = document.bufferViews?.[accessor?.bufferView]
if (!accessor || accessor.type !== type || accessor.componentType !== 5126 || accessor.sparse || !bytes || bytes.buffer !== 0 || bytes.byteStride || !Number.isInteger(accessor.count) || accessor.count < 1 || accessor.count > 100000) {
throw new Error('Unsupported animation accessor')
}
const relative = accessor.byteOffset ?? 0
const offset = bytes.byteOffset ?? 0
if (![relative, offset, bytes.byteLength].every(value => Number.isInteger(value) && value >= 0)) throw new Error('Invalid animation buffer view')
const start = offset + relative
const size = accessor.count * components * 4
if (start < 0 || start % 4 || relative < 0 || relative + size > bytes.byteLength || start + size > binary.byteLength) throw new Error('Animation accessor exceeds buffer')
const values = new Float32Array(binary, start, accessor.count * components)
if (!values.every(Number.isFinite)) throw new Error('Animation contains non-finite values')
return values
}
const clip = document.animations?.[0]
if (!clip?.channels?.length) throw new Error('GLB has no animation tracks')
const tracks = clip.channels.map(channel => {
const sampler = clip.samplers?.[channel.sampler]
const path = channel.target?.path
if (!sampler || !nodes[channel.target?.node] || !['rotation', 'translation'].includes(path) || !['LINEAR', 'STEP'].includes(sampler.interpolation || 'LINEAR')) {
throw new Error('Unsupported animation track')
}
const times = readAccessor(sampler.input, 'SCALAR', 1)
if (times[0] < 0 || times.some((time, index) => index > 0 && time <= times[index-1])) throw new Error('Invalid animation timeline')
const components = path === 'rotation' ? 4 : 3
const values = readAccessor(sampler.output, path === 'rotation' ? 'VEC4' : 'VEC3', components)
if (values.length !== times.length * components) throw new Error('Animation track length mismatch')
return { node: channel.target.node, path, times, values, components, interpolation: sampler.interpolation }
})
return { nodes, order, tracks, duration: Math.max(...tracks.map(track => track.times.at(-1))) }
}
function multiplyQuaternion(a, b) {
return [a[3]*b[0]+a[0]*b[3]+a[1]*b[2]-a[2]*b[1], a[3]*b[1]-a[0]*b[2]+a[1]*b[3]+a[2]*b[0], a[3]*b[2]+a[0]*b[1]-a[1]*b[0]+a[2]*b[3], a[3]*b[3]-a[0]*b[0]-a[1]*b[1]-a[2]*b[2]]
}
function rotate(q, vector) {
const result = multiplyQuaternion(multiplyQuaternion(q, [...vector, 0]), [-q[0], -q[1], -q[2], q[3]])
return result.slice(0, 3)
}
export function animationPose(animation, time) {
const local = animation.nodes.map(node => ({ translation: node.translation, rotation: node.rotation }))
for (const track of animation.tracks) {
let frame = 0
while (frame + 1 < track.times.length && track.times[frame+1] <= time) frame++
const next = Math.min(frame+1, track.times.length-1)
const fraction = next === frame || track.interpolation === 'STEP' ? 0 : Math.max(0, (time-track.times[frame]) / (track.times[next]-track.times[frame]))
const a = track.values.subarray(frame*track.components, (frame+1)*track.components)
const b = track.values.subarray(next*track.components, (next+1)*track.components)
const sign = track.path === 'rotation' && a.reduce((sum, value, index) => sum + value*b[index], 0) < 0 ? -1 : 1
let value = Array.from(a, (component, index) => component*(1-fraction) + sign*b[index]*fraction)
if (track.path === 'rotation') {
const length = Math.hypot(...value)
if (length < 1e-8) throw new Error('Invalid animation quaternion')
value = value.map(component => component/length)
}
local[track.node][track.path] = value
}
const positions = [], rotations = []
for (const index of animation.order) {
const parent = animation.nodes[index].parent
if (parent < 0) {
positions[index] = local[index].translation
rotations[index] = local[index].rotation
} else {
rotations[index] = multiplyQuaternion(rotations[parent], local[index].rotation)
const offset = rotate(rotations[parent], local[index].translation)
positions[index] = offset.map((value, axis) => value+positions[parent][axis])
}
}
return positions
}
addons/studio/ui/src/utils/diarization.js
+89
View File
@@ -0,0 +1,89 @@
// What a diarization run says, as numbers and as the files people already use
// for it. Plain functions with no React, so they can be tested with `node --test`.
//
// A segment is { start, end, speaker, label, name?, text? } in seconds. `label`
// is the stable key of a speaker inside one run; `speaker` is the id the server
// prints (SPEAKER_00).
const num = (v) => (Number.isFinite(Number(v)) ? Number(v) : 0)
// m:ss, or h:mm:ss for a long recording.
export function clock(seconds) {
const total = Math.max(0, Math.round(num(seconds)))
const h = Math.floor(total / 3600)
const m = Math.floor((total % 3600) / 60)
const s = total % 60
return h > 0 ? `${h}:${String(m).padStart(2, '0')}:${String(s).padStart(2, '0')}` : `${m}:${String(s).padStart(2, '0')}`
}
export function runLength(result) {
return Math.max(0, ...(result?.segments || []).map(s => num(s.end)))
}
// One row per speaker, in order of first speech: how long they spoke and what
// share of all speech that is. Overlapping speech counts for each speaker.
export function speakerRows(result) {
const segments = result?.segments || []
const rows = new Map()
for (const seg of segments) {
const key = String(seg.label ?? seg.speaker)
if (!rows.has(key)) rows.set(key, { label: key, id: seg.speaker ?? key, name: seg.name || '', seconds: 0, segments: [] })
const row = rows.get(key)
row.seconds += Math.max(0, num(seg.end) - num(seg.start))
if (!row.name && seg.name) row.name = seg.name
row.segments.push(seg)
}
for (const s of result?.speakers || []) {
const key = String(s.label)
if (!rows.has(key)) rows.set(key, { label: key, id: s.id ?? key, name: s.name || '', seconds: 0, segments: [] })
else if (s.name && !rows.get(key).name) rows.get(key).name = s.name
}
const total = [...rows.values()].reduce((sum, r) => sum + r.seconds, 0)
return [...rows.values()].map((r, index) => ({ ...r, index, share: total > 0 ? r.seconds / total : 0 }))
}
// Tick marks for the timeline axis: about `count` round steps across the run.
export function axisTicks(length, count = 5) {
if (!(length > 0)) return []
const steps = [1, 2, 5, 10, 15, 30, 60, 120, 300, 600, 900, 1800, 3600]
const raw = length / Math.max(1, count - 1)
const step = steps.find(s => s >= raw * 0.7) || steps[steps.length - 1]
const ticks = []
for (let t = 0; t <= length + 1e-6; t += step) ticks.push({ at: t, pct: (t / length) * 100 })
return ticks
}
// RTTM, the line format diarization tools exchange.
export function toRttm(result, fileName = 'recording') {
const id = String(fileName).replace(/\.[^.]*$/, '').replace(/\s+/g, '_') || 'recording'
const rows = speakerRows(result)
const names = new Map(rows.map(r => [r.label, r.name || r.id]))
return (result?.segments || [])
.map((s) => {
const who = String(names.get(String(s.label ?? s.speaker)) || s.speaker).replace(/\s+/g, '_')
const dur = Math.max(0, num(s.end) - num(s.start))
return `SPEAKER ${id} 1 ${num(s.start).toFixed(3)} ${dur.toFixed(3)} <NA> <NA> ${who} <NA> <NA>`
})
.join('\n') + '\n'
}
function srtTime(seconds) {
const ms = Math.round(Math.max(0, num(seconds)) * 1000)
const h = Math.floor(ms / 3600000)
const m = Math.floor((ms % 3600000) / 60000)
const s = Math.floor((ms % 60000) / 1000)
const pad = (n, w = 2) => String(n).padStart(w, '0')
return `${pad(h)}:${pad(m)}:${pad(s)},${pad(ms % 1000, 3)}`
}
// SRT of the segments that have text. Empty when the run carried no text.
export function toSrt(result) {
const rows = speakerRows(result)
const names = new Map(rows.map(r => [r.label, r.name || r.id]))
return (result?.segments || [])
.filter(s => String(s.text || '').trim())
.map((s, i) => `${i + 1}\n${srtTime(s.start)} --> ${srtTime(s.end)}\n${names.get(String(s.label ?? s.speaker)) || s.speaker}: ${String(s.text).trim()}\n`)
.join('\n')
}
export const hasText = (result) => (result?.segments || []).some(s => String(s.text || '').trim())
addons/studio/ui/src/utils/fft.js
+47
View File
@@ -0,0 +1,47 @@
// Minimal in-place iterative radix-2 Cooley–Tukey FFT.
//
// The AudioTransform spectrogram only needs forward transforms of short real
// frames (≤2048 samples), so a compact ~30-line implementation beats pulling
// in a dependency and shipping it in the bundle. `re` and `im` are mutated in
// place; `n = re.length` must be a power of two (the caller picks fftSize).
export function fftRadix2(re, im) {
const n = re.length
if (n <= 1) return
// Bit-reversal permutation: reorder samples so the butterfly stage below can
// run in place.
for (let i = 1, j = 0; i < n; i++) {
let bit = n >> 1
for (; j & bit; bit >>= 1) j ^= bit
j ^= bit
if (i < j) {
const tr = re[i]; re[i] = re[j]; re[j] = tr
const ti = im[i]; im[i] = im[j]; im[j] = ti
}
}
// Butterflies, doubling the transform length each pass.
for (let len = 2; len <= n; len <<= 1) {
const half = len >> 1
const ang = (-2 * Math.PI) / len
const wpr = Math.cos(ang)
const wpi = Math.sin(ang)
for (let i = 0; i < n; i += len) {
let wr = 1
let wi = 0
for (let k = 0; k < half; k++) {
const a = i + k
const b = a + half
const tr = wr * re[b] - wi * im[b]
const ti = wr * im[b] + wi * re[b]
re[b] = re[a] - tr
im[b] = im[a] - ti
re[a] += tr
im[a] += ti
const nwr = wr * wpr - wi * wpi
wi = wr * wpi + wi * wpr
wr = nwr
}
}
}
}
addons/studio/ui/src/utils/glb.js
+144
View File
@@ -0,0 +1,144 @@
// Minimal GLB (binary glTF 2.0) parser for the two forms trellis2cpp's
// t2_bake_glb emits (see mesh_export.cpp in localai-org/trellis2cpp):
//
// Form B (default, dense vertex PBR):
// POSITION (VEC3 f32), NORMAL (VEC3 f32),
// COLOR_0 (VEC4 u16 normalized, LINEAR color + alpha),
// _METALLIC_ROUGHNESS (VEC2 u8 normalized: metallic, roughness),
// indices (SCALAR u32); material carries average metallic/roughness factors.
//
// Form A (opt-in via T2GLB_XATLAS, UV atlas):
// POSITION/NORMAL/TEXCOORD_0 f32 + u32 indices,
// baseColorTexture (sRGB PNG) + metallicRoughnessTexture
// (glTF convention: G=roughness, B=metallic).
//
// Deliberately unsupported (the baker never emits them; anything else throws
// so the page can show the error while the download button still works):
// sparse accessors, interleaved bufferViews (byteStride), Draco, multiple
// meshes/primitives.
const GLB_MAGIC = 0x46546c67
const CHUNK_JSON = 0x4e4f534a
const CHUNK_BIN = 0x004e4942
const COMPONENT_ARRAYS = {
5121: Uint8Array,
5123: Uint16Array,
5125: Uint32Array,
5126: Float32Array,
}
const TYPE_SIZES = { SCALAR: 1, VEC2: 2, VEC3: 3, VEC4: 4 }
export function parseGlb(buf) {
if (!(buf instanceof ArrayBuffer) || buf.byteLength < 20) throw new Error('not a GLB file')
const dv = new DataView(buf)
if (dv.getUint32(0, true) !== GLB_MAGIC) throw new Error('not a GLB file')
let offset = 12
let json = null
let binOffset = -1
let binLength = 0
while (offset + 8 <= buf.byteLength) {
const len = dv.getUint32(offset, true)
const type = dv.getUint32(offset + 4, true)
const payload = offset + 8
if (type === CHUNK_JSON && !json) {
json = JSON.parse(new TextDecoder().decode(new Uint8Array(buf, payload, len)))
} else if (type === CHUNK_BIN && binOffset < 0) {
binOffset = payload
binLength = len
}
offset = payload + len + ((4 - (len % 4)) % 4)
}
if (!json) throw new Error('GLB has no JSON chunk')
const accessor = (index) => {
const a = json.accessors?.[index]
if (!a) throw new Error(`missing accessor ${index}`)
if (a.sparse) throw new Error('sparse accessors not supported')
const view = json.bufferViews?.[a.bufferView]
if (!view) throw new Error(`missing bufferView ${a.bufferView}`)
if (view.byteStride) throw new Error('interleaved bufferViews not supported')
const ArrayType = COMPONENT_ARRAYS[a.componentType]
const size = TYPE_SIZES[a.type]
if (!ArrayType || !size) throw new Error(`unsupported accessor layout ${a.componentType}/${a.type}`)
const start = binOffset + (view.byteOffset || 0) + (a.byteOffset || 0)
if (binOffset < 0 || start + a.count * size * ArrayType.BYTES_PER_ELEMENT > binOffset + binLength) {
throw new Error('accessor outside the BIN chunk')
}
return {
array: new ArrayType(buf, start, a.count * size),
size,
normalized: !!a.normalized,
min: a.min,
max: a.max,
count: a.count,
}
}
const prim = json.meshes?.[0]?.primitives?.[0]
if (!prim) throw new Error('GLB has no mesh')
const attrs = prim.attributes || {}
if (attrs.POSITION === undefined) throw new Error('GLB mesh has no POSITION attribute')
const position = accessor(attrs.POSITION)
const normal = attrs.NORMAL !== undefined ? accessor(attrs.NORMAL) : null
let indices
if (prim.indices !== undefined) {
const idx = accessor(prim.indices)
indices = idx.array instanceof Uint32Array ? idx.array : Uint32Array.from(idx.array)
} else {
indices = new Uint32Array(position.count)
for (let i = 0; i < indices.length; i++) indices[i] = i
}
const color0 = attrs.COLOR_0 !== undefined ? accessor(attrs.COLOR_0) : null
const metalRough = attrs._METALLIC_ROUGHNESS !== undefined ? accessor(attrs._METALLIC_ROUGHNESS) : null
const uv = attrs.TEXCOORD_0 !== undefined ? accessor(attrs.TEXCOORD_0) : null
const material = json.materials?.[prim.material]?.pbrMetallicRoughness || {}
const imageBytes = (textureIndex) => {
if (textureIndex === undefined) return null
const source = json.textures?.[textureIndex]?.source
const view = json.bufferViews?.[json.images?.[source]?.bufferView]
if (!view) return null
return new Uint8Array(buf, binOffset + (view.byteOffset || 0), view.byteLength)
}
// POSITION min/max are mandatory in glTF, but compute a fallback so a
// technically-invalid file still frames correctly.
let bboxMin = position.min
let bboxMax = position.max
if (!bboxMin || !bboxMax) {
bboxMin = [Infinity, Infinity, Infinity]
bboxMax = [-Infinity, -Infinity, -Infinity]
for (let i = 0; i < position.array.length; i += 3) {
for (let k = 0; k < 3; k++) {
const v = position.array[i + k]
if (v < bboxMin[k]) bboxMin[k] = v
if (v > bboxMax[k]) bboxMax[k] = v
}
}
}
return {
positions: position.array,
normals: normal ? normal.array : null,
indices,
color0,
metalRough,
uv: uv ? uv.array : null,
baseColorPng: imageBytes(material.baseColorTexture?.index),
metalRoughPng: imageBytes(material.metallicRoughnessTexture?.index),
material: {
metallicFactor: material.metallicFactor ?? 1,
roughnessFactor: material.roughnessFactor ?? 1,
},
bboxMin,
bboxMax,
nVerts: position.count,
nTris: Math.floor(indices.length / 3),
}
}
addons/studio/ui/src/utils/mediaHistoryStore.js
+29
View File
@@ -0,0 +1,29 @@
// Where the Studio workspaces keep their history, as plain functions with no
// React in them so they can be tested on their own.
export const MEDIA_STORAGE_KEYS = {
image: 'localai_image_history',
video: 'localai_video_history',
tts: 'localai_tts_history',
sound: 'localai_sound_history',
'audio-transform': 'localai_audio_transform_history',
// Diarization kept no history before the Studio front page; it records the
// file name, the model and a speaker count here, never the recording.
diarization: 'localai_diarization_history',
}
export const FAVOURITES_KEY = 'localai_studio_favourites'
// Forget everything the Studio front page shows: every media history list and
// the favourites. 3D lives in IndexedDB, so the caller clears that with
// use3DHistory().clearAll(). Returns how many stored lists were removed.
export function clearAllMediaHistory() {
let removed = 0
for (const key of [...Object.values(MEDIA_STORAGE_KEYS), FAVOURITES_KEY]) {
try {
if (localStorage.getItem(key) !== null) removed += 1
localStorage.removeItem(key)
} catch { /* storage unavailable: nothing to clear */ }
}
return removed
}
addons/studio/ui/src/utils/studioWork.js
+508
View File
@@ -0,0 +1,508 @@
// The data model behind the Studio front page.
//
// Studio has seven workspaces and each one already records what it made, in its
// own store: five browser-storage lists (useMediaHistory), 3D in IndexedDB
// (use3DHistory), and, from this change, one for diarization. This module turns
// those into one list of "work items", links items that were made from each
// other, and lays the links out as a lineage board. It is plain functions with
// no React and no storage, so it can be tested with `node --test`.
//
// Parent links are written by the workspace that makes the result: `parentId`
// is the id of the result it was made from and `edge` says how. Results made
// before the links existed have none and stand alone.
export const TYPE_ORDER = ['images', 'video', 'threed', 'tts', 'sound', 'transform', 'diarization']
// What the front page needs to know about each workspace that the workspaces
// themselves do not export:
// media the useMediaHistory key (null for 3D, which has its own store)
// input 'text' when the workspace starts from a sentence, 'file' when it
// needs a picture or a recording
// source the kind of file it can start from: 'image', 'audio' or null
// sizes the sizes its form offers, when it has a size control
// tag the gallery tag used to suggest a model when none is installed
export const TYPE_INFO = {
images: { media: 'image', kind: 'image', input: 'text', source: 'image', sizes: ['256x256', '512x512', '768x768', '1024x1024'], defaultSize: '512x512', maxCount: 4, tag: 'text-to-image' },
video: { media: 'video', kind: 'video', input: 'text', source: 'image', sizes: ['256x256', '512x512', '768x768', '1024x1024', '832x480', '1280x720'], defaultSize: '512x512', tag: 'text-to-video' },
threed: { media: null, kind: 'mesh', input: 'file', source: 'image', needsSource: true, tag: 'image-to-3d' },
tts: { media: 'tts', kind: 'audio', input: 'text', source: null, tag: 'tts' },
sound: { media: 'sound', kind: 'audio', input: 'text', source: null, tag: 'music' },
transform: { media: 'audio-transform', kind: 'audio', input: 'file', source: 'audio', needsSource: true, tag: 'audio-transform' },
diarization: { media: 'diarization', kind: 'speakers', input: 'file', source: 'audio', needsSource: true, tag: 'diarization' },
}
export const TYPE_ICON = {
images: 'image', video: 'video', threed: 'cube', tts: 'headphones', sound: 'music', transform: 'waveform', diarization: 'users',
}
const TYPE_BY_MEDIA = Object.fromEntries(
Object.entries(TYPE_INFO).filter(([, v]) => v.media).map(([key, v]) => [v.media, key]),
)
export const EDGE_KINDS = ['upscale', 'take', 'animate', 'to-3d', 'variation', 'transform', 'diarize']
// What each kind of result can be sent on to. `supported` is whether the
// destination workspace accepts that result as a starting point today. The
// unsupported steps are kept in the list on purpose: the lineage view shows
// them disabled with the reason instead of hiding what a person might expect.
// hint names the reason: the destination takes no `source`
export const NEXT_STEPS = {
images: [
{ to: 'video', edge: 'animate', supported: true },
{ to: 'threed', edge: 'to-3d', supported: true },
{ to: 'images', edge: 'variation', supported: true },
],
video: [
{ to: 'sound', edge: 'soundtrack', supported: false, source: 'video' },
],
threed: [
{ to: 'video', edge: 'turntable', supported: false, source: 'mesh' },
],
tts: [
{ to: 'transform', edge: 'transform', supported: true },
{ to: 'diarization', edge: 'diarize', supported: true },
],
sound: [
{ to: 'transform', edge: 'transform', supported: true },
{ to: 'diarization', edge: 'diarize', supported: true },
],
transform: [
{ to: 'diarization', edge: 'diarize', supported: true },
],
diarization: [],
}
// Every kind of result can be run again as a new take, but a few need the
// original input and the workspace no longer has it.
// threed the picture is kept only as a small thumbnail
// diarization the recording is not kept
// Everything else carries what it needs in the stored entry.
export function canTake(item) {
if (!item) return { ok: false }
if (item.type === 'threed') return { ok: false, reason: 'noInput' }
if (item.type === 'diarization') return { ok: false, reason: 'noInput' }
if (item.type === 'transform' && !item.inputUrl) return { ok: false, reason: 'noInput' }
return { ok: true }
}
// --- Suggesting a type from the words ---------------------------------------
// First match wins, so the narrow cases sit above the broad ones. No network,
// no model: a short list of words people use for each kind of work. A wrong
// guess costs one click, so it only ever suggests and never switches.
const SUGGESTIONS = [
['diarization', /\b(who (spoke|said|talks?)|speakers?|diari[sz]\w*)\b/i],
['transform', /\b(clean up|denoise|noise reduction|change my voice|voice conver\w*|remove (the )?background noise)\b/i],
['threed', /\b(3-?d|mesh|glb|turn it into an? object)\b/i],
['video', /\b(video|clip|footage|timelapse|time-lapse|dolly|pan(ning)?|tracking shot|drone shot|camera move|animate)\b/i],
['sound', /\b(music|ambient|ambience|lo-?fi|soundtrack|beat|piano|jingle|sound effects?|melody|song)\b/i],
['tts', /\b(read (it |this )?(aloud|out)|narrat\w*|voice-?over|speak|say|text to speech|tts)\b/i],
['images', /\b(image|picture|photo(graph)?|painting|illustration|poster|portrait|logo|drawing|sketch|render|wallpaper|icon)\b/i],
]
export const MIN_SUGGEST_LENGTH = 8
export function suggestType(text) {
const words = String(text || '').trim()
if (words.length < MIN_SUGGEST_LENGTH) return null
for (const [type, pattern] of SUGGESTIONS) {
if (pattern.test(words)) return type
}
return null
}
// --- Work items -------------------------------------------------------------
export function parseSize(size) {
const m = /^(\d{2,5})\s*[x×]\s*(\d{2,5})$/i.exec(String(size || '').trim())
return m ? { width: Number(m[1]), height: Number(m[2]) } : null
}
// Width over height for a tile. Only what the entry recorded: an entry with no
// size is drawn square rather than guessed.
export function aspectOf(item) {
const output = item?.params?.outputDimensions
const size = output?.width > 0 && output?.height > 0 ? output : parseSize(item?.params?.size)
return size ? size.width / size.height : 1
}
const clip = (value, max) => {
const s = String(value ?? '')
return s.length > max ? s.slice(0, max) : s
}
// One normalised item from a stored history entry. `type` is the Studio key.
export function toWorkItem(type, entry) {
const results = Array.isArray(entry?.results) ? entry.results : []
const outputs = results.filter(r => r && r.url && (!r.kind || r.kind === 'output'))
const input = results.find(r => r && r.url && r.kind === 'input')
const title = type === 'threed'
? (entry.label || entry.inputs?.prompt?.data || entry.name || '')
: (entry.prompt || '')
return {
id: String(entry.id),
type,
title: clip(title, 2000),
model: entry.model || '',
createdAt: Number(entry.createdAt) || 0,
parentId: entry.parentId ? String(entry.parentId) : null,
edge: entry.edge || null,
url: outputs[0]?.url || '',
urls: outputs.map(r => r.url),
inputUrl: input?.url || '',
thumb: type === 'threed' ? (entry.inputThumb || '') : '',
params: entry.params && typeof entry.params === 'object' ? entry.params : {},
elapsedMs: Number(entry.elapsedMs) || 0,
outputType: entry.outputType || '',
}
}
// Every stored entry as one list, newest first. `media` is the object that
// readAllMediaHistory() returns; `threeD` is the array use3DHistory() holds.
export function collectWork(media, threeD, { favourites = [] } = {}) {
const fav = new Set(favourites)
const out = []
for (const [key, entries] of Object.entries(media || {})) {
const type = TYPE_BY_MEDIA[key]
if (!type || !Array.isArray(entries)) continue
for (const entry of entries) {
if (entry && entry.id != null) out.push(toWorkItem(type, entry))
}
}
for (const entry of Array.isArray(threeD) ? threeD : []) {
if (entry && entry.id != null) out.push(toWorkItem('threed', entry))
}
return out
.map(item => ({ ...item, favourite: fav.has(item.id) }))
.sort((a, b) => b.createdAt - a.createdAt)
}
export function countByType(items) {
const counts = { all: items.length, favourites: 0 }
for (const key of TYPE_ORDER) counts[key] = 0
for (const item of items) {
counts[item.type] += 1
if (item.favourite) counts.favourites += 1
}
return counts
}
export function filterWork(items, filter) {
if (filter === 'favourites') return items.filter(i => i.favourite)
if (TYPE_INFO[filter]) return items.filter(i => i.type === filter)
return items
}
// --- Projects ---------------------------------------------------------------
// Results linked by a parent that still exists form one project. A link to a
// result that has since been deleted is ignored, so the child stands as a root.
function parentMap(items) {
const ids = new Set(items.map(i => i.id))
const parents = new Map()
// A link that would close a loop is dropped; storage can be edited by hand.
const loops = (child, parent) => {
for (let cur = parent, n = 0; cur && n < 1000; cur = parents.get(cur), n += 1) if (cur === child) return true
return false
}
for (const item of items) {
if (item.parentId && item.parentId !== item.id && ids.has(item.parentId) && !loops(item.id, item.parentId)) {
parents.set(item.id, item.parentId)
}
}
return parents
}
function components(items) {
const parents = parentMap(items)
const root = new Map()
const find = (id) => {
let cur = id
const seen = new Set()
while (root.has(cur) && root.get(cur) !== cur && !seen.has(cur)) { seen.add(cur); cur = root.get(cur) }
return cur
}
for (const item of items) root.set(item.id, item.id)
for (const [child, parent] of parents) {
const a = find(child)
const b = find(parent)
if (a !== b) root.set(a, b)
}
const groups = new Map()
for (const item of items) {
const key = find(item.id)
if (!groups.has(key)) groups.set(key, [])
groups.get(key).push(item)
}
return [...groups.values()]
}
// The result that stands for a project on a tile: the newest one with a picture
// to show, else the newest.
function coverOf(members) {
const byNew = [...members].sort((a, b) => b.createdAt - a.createdAt)
return byNew.find(m => m.type === 'images' || m.thumb) || byNew.find(m => m.type === 'video') || byNew[0]
}
// The masonry's tiles, newest activity first. A set of two or more linked
// results becomes one project tile; the rest stay single.
export function groupWork(items) {
const tiles = components(items).map((members) => {
const sorted = [...members].sort((a, b) => a.createdAt - b.createdAt)
if (members.length === 1) return { kind: 'single', id: members[0].id, item: members[0], latest: members[0].createdAt }
const root = sorted.find(m => !m.parentId || !members.some(o => o.id === m.parentId)) || sorted[0]
return {
kind: 'project',
id: root.id,
items: sorted,
cover: coverOf(members),
title: root.title,
latest: sorted[sorted.length - 1].createdAt,
favourite: members.some(m => m.favourite),
}
})
return tiles.sort((a, b) => b.latest - a.latest)
}
// The project a result belongs to, as the same list the tile holds.
export function projectOf(items, id) {
const group = components(items).find(members => members.some(m => m.id === id))
if (!group) return null
return [...group].sort((a, b) => a.createdAt - b.createdAt)
}
// --- Lineage board ----------------------------------------------------------
export const BOARD = { nodeW: 156, srcH: 128, resH: 134, gapX: 96, gapY: 22, pad: 28 }
// The node an item's edge starts from. A result with no parent grows from its
// own prompt or file. A take shares the origin of the result it repeats, so two
// takes of one prompt hang off the same prompt node. Anything else grows from
// its parent result.
function originOf(item, byId, parents) {
const parent = parents.has(item.id) ? byId.get(parents.get(item.id)) : null
if (!parent) return `src:${item.id}`
if (item.edge === 'take') return originOf(parent, byId, parents)
return `res:${parent.id}`
}
// Nodes, edges and the chain of the selection, positioned on a grid. `ghost` is
// an optional dashed node hung off the selected result: { id, from, type,
// label, missing }. Positions are plain numbers in board pixels.
export function layoutLineage(members, { selectedId = null, ghost = null } = {}) {
const byId = new Map(members.map(m => [m.id, m]))
const parents = parentMap(members)
const nodes = new Map()
const edges = []
// Sources first, so a take can find the prompt it shares whatever the order.
for (const m of members) {
if (parents.has(m.id)) continue
const info = TYPE_INFO[m.type]
nodes.set(`src:${m.id}`, {
id: `src:${m.id}`, kind: 'source', input: info.input, type: m.type,
title: info.input === 'text' ? m.title : '', createdAt: m.createdAt, children: [],
})
}
for (const m of members) {
const from = originOf(m, byId, parents)
nodes.set(`res:${m.id}`, { id: `res:${m.id}`, kind: 'result', item: m, children: [] })
edges.push({ id: `${from}>res:${m.id}`, from, to: `res:${m.id}`, label: parents.has(m.id) ? m.edge : null })
}
if (ghost) {
nodes.set(ghost.id, { id: ghost.id, kind: 'ghost', ghost, children: [] })
edges.push({ id: `${ghost.from}>${ghost.id}`, from: ghost.from, to: ghost.id, label: ghost.edge, ghost: true })
}
for (const e of edges) nodes.get(e.from)?.children.push(e.to)
// Children in the order they were made.
const stamp = (id) => {
const n = nodes.get(id)
return n.kind === 'result' ? n.item.createdAt : n.kind === 'ghost' ? Infinity : n.createdAt
}
for (const n of nodes.values()) n.children.sort((a, b) => stamp(a) - stamp(b))
// Depth is distance from the root node; rows come from a leaf-counting walk.
const hasParent = new Set(edges.map(e => e.to))
const roots = [...nodes.values()].filter(n => !hasParent.has(n.id)).sort((a, b) => stamp(a.id) - stamp(b.id))
let row = 0
const place = (id, depth) => {
const n = nodes.get(id)
n.depth = depth
if (n.children.length === 0) { n.row = row; row += 1; return }
n.children.forEach(c => place(c, depth + 1))
const kids = n.children.map(c => nodes.get(c).row)
n.row = (Math.min(...kids) + Math.max(...kids)) / 2
}
roots.forEach(r => place(r.id, 0))
const { nodeW, srcH, resH, gapX, gapY, pad } = BOARD
const rowH = Math.max(srcH, resH) + gapY
let width = 0
let height = 0
for (const n of nodes.values()) {
const h = n.kind === 'source' ? srcH : resH
n.w = nodeW
n.h = h
n.x = pad + n.depth * (nodeW + gapX)
n.y = pad + n.row * rowH + (rowH - gapY - h) / 2
width = Math.max(width, n.x + n.w + pad)
height = Math.max(height, n.y + n.h + pad)
}
// The chain: the selection and everything above it, back to the root.
const parentOf = new Map(edges.map(e => [e.to, e.from]))
const chain = new Set()
let cur = selectedId ? `res:${selectedId}` : null
while (cur && nodes.has(cur) && !chain.has(cur)) { chain.add(cur); cur = parentOf.get(cur) }
if (ghost && chain.has(ghost.from)) chain.add(ghost.id)
const placed = edges.map(e => {
const a = nodes.get(e.from)
const b = nodes.get(e.to)
const x1 = a.x + a.w
const y1 = a.y + a.h * 0.42
const x2 = b.x
const y2 = b.y + b.h * 0.42
const mid = (x1 + x2) / 2
return {
...e, x1, y1, x2, y2,
d: `M${x1} ${y1} C${mid} ${y1} ${mid} ${y2} ${x2} ${y2}`,
lx: mid, ly: (y1 + y2) / 2,
chain: chain.has(e.from) && chain.has(e.to),
}
})
return { nodes: [...nodes.values()], edges: placed, chain, width: Math.max(width, 320), height: Math.max(height, 200) }
}
// The result next to the selection in a direction, for walking the board with
// the arrow keys. Left and right follow the lineage, up and down walk the
// results in the same column.
export function neighbour(layout, selectedId, direction) {
const results = layout.nodes.filter(n => n.kind === 'result')
const here = results.find(n => n.item.id === selectedId)
if (!here) return results[0]?.item.id || null
if (direction === 'left') {
const edge = layout.edges.find(e => e.to === here.id)
const from = edge && layout.nodes.find(n => n.id === edge.from)
if (from?.kind === 'result') return from.item.id
return selectedId
}
if (direction === 'right') {
const child = here.children.map(id => layout.nodes.find(n => n.id === id)).find(n => n?.kind === 'result')
return child ? child.item.id : selectedId
}
const column = results.filter(n => n.depth === here.depth).sort((a, b) => a.y - b.y)
const i = column.findIndex(n => n.id === here.id)
const next = column[i + (direction === 'down' ? 1 : -1)]
return next ? next.item.id : selectedId
}
// The dashed "suggested" step: the first step the destination accepts and a
// model is installed for, else the first one it accepts. `installed` is a set of
// type keys that have a model.
export function suggestNext(type, installed) {
const steps = (NEXT_STEPS[type] || []).filter(s => s.supported)
return steps.find(s => installed.has(s.to)) || steps[0] || null
}
// --- Hand-off ---------------------------------------------------------------
// The workspaces read these query parameters. Nothing else is carried: a prompt,
// a model, a size, a count, the id of the result it grows from and how.
export function handoffPath(type, { prompt, model, size, count, from, edge } = {}) {
const q = new URLSearchParams()
if (prompt && String(prompt).trim()) q.set('prompt', String(prompt).trim())
if (model) q.set('model', model)
if (size) q.set('size', size)
if (count && Number(count) > 1) q.set('n', String(count))
if (from) q.set('from', from)
if (edge) q.set('edge', edge)
const qs = q.toString()
return `/app/studio/${type}${qs ? `?${qs}` : ''}`
}
export function readHandoff(params) {
const get = (k) => params.get(k) || ''
const n = parseInt(get('n'), 10)
return {
prompt: get('prompt'),
model: get('model'),
size: get('size'),
count: Number.isFinite(n) && n > 0 ? n : 0,
from: get('from'),
edge: get('edge'),
}
}
// The file a destination starts from when it grows out of `item`.
export function sourceUrlFor(item, edge) {
if (!item) return ''
if (edge === 'take') return item.inputUrl || ''
return item.url || ''
}
// --- Favourites -------------------------------------------------------------
export const MAX_FAVOURITES = 500
// Add or remove an id, newest last, bounded. Returns a new array.
export function toggleId(ids, id) {
if (ids.includes(id)) return ids.filter(x => x !== id)
const next = [...ids, id]
return next.length > MAX_FAVOURITES ? next.slice(next.length - MAX_FAVOURITES) : next
}
// Keep only ids that still name a stored result.
export function pruneIds(ids, items) {
const live = new Set(items.map(i => i.id))
return ids.filter(id => live.has(id))
}
// --- Masonry ----------------------------------------------------------------
// Number of columns for a container width.
export function columnCount(width) {
if (!(width > 0)) return 1
return Math.max(1, Math.min(4, Math.floor(width / 270)))
}
// Estimated height of a tile in column units, used only to balance columns. A
// picture tile is as tall as its aspect makes it; a sound tile is a short bar.
export function tileWeight(tile) {
const item = tile.kind === 'project' ? tile.cover : tile.item
const kind = TYPE_INFO[item.type].kind
let body = 1
if (kind === 'image' || kind === 'video') body = Math.min(1.8, Math.max(0.55, 1 / aspectOf(item)))
else if (kind === 'audio' || kind === 'speakers') body = 0.45
else body = 0.9
return body + (tile.kind === 'project' ? 0.5 : 0.3)
}
// Deal tiles into columns, each to the shortest column so far. Order is kept
// within a column so the newest sit at the top.
export function dealColumns(tiles, count) {
const cols = Array.from({ length: count }, () => ({ h: 0, tiles: [] }))
for (const tile of tiles) {
const target = cols.reduce((a, b) => (b.h < a.h ? b : a))
target.tiles.push(tile)
target.h += tileWeight(tile)
}
return cols.map(c => c.tiles)
}
// A repeatable pseudo-waveform for an audio tile, so one tile always looks the
// same. It is a placeholder, not the sound.
export function waveBars(id, count = 48) {
let h = 2166136261
for (let i = 0; i < String(id).length; i++) { h ^= String(id).charCodeAt(i); h = Math.imul(h, 16777619) }
const bars = []
for (let i = 0; i < count; i++) {
h ^= h << 13; h ^= h >>> 17; h ^= h << 5
const r = ((h >>> 0) % 1000) / 1000
const envelope = 0.45 + 0.55 * Math.sin((i / count) * Math.PI)
bars.push(Math.round((0.15 + r * 0.85 * envelope) * 100) / 100)
}
return bars
}
addons/studio/ui/src/utils/upscale.js
+39
View File
@@ -0,0 +1,39 @@
// SPDX-License-Identifier: MIT
import { CAP_UPSCALE } from './capabilities.js'
export function upscaleModels(models) {
return models.filter(m => !m.disabled && m.capabilities?.includes(CAP_UPSCALE) &&
Number.isFinite(m.upscaleScale) && m.upscaleScale > 0)
.map(({ id, upscaleScale }) => ({ id, upscaleScale }))
}
export async function sourceImageFile(source, base = window.location.href, fetcher = fetch) {
if (source instanceof File) return source
const url = new URL(source, base)
if (url.origin !== new URL(base).origin || !['http:', 'https:'].includes(url.protocol)) {
throw new Error('Source image must be same-origin')
}
const response = await fetcher(url.href, { credentials: 'same-origin', mode: 'same-origin', redirect: 'error' })
if (!response.ok) throw new Error(`Image fetch failed (HTTP ${response.status})`)
if (!response.headers.get('content-type')?.toLowerCase().startsWith('image/')) throw new Error('Source is not an image')
const blob = await response.blob()
return new File([blob], url.pathname.split('/').pop() || 'source.png', { type: blob.type })
}
export function imageDimensions(source) {
return new Promise((resolve, reject) => {
const objectUrl = source instanceof Blob ? URL.createObjectURL(source) : null
const image = new Image()
const finish = (error) => {
clearTimeout(timer)
image.onload = image.onerror = null
if (objectUrl) URL.revokeObjectURL(objectUrl)
if (error || !image.naturalWidth || !image.naturalHeight) reject(new Error('Could not decode image dimensions'))
else resolve({ width: image.naturalWidth, height: image.naturalHeight })
}
const timer = setTimeout(() => finish(true), 30000)
image.onload = () => finish(false)
image.onerror = () => finish(true)
image.src = objectUrl || source
})
}
addons/studio/ui/src/utils/voiceEnrollments.js
+20
View File
@@ -0,0 +1,20 @@
// SPDX-License-Identifier: MIT
const ENROLL_KEY = 'localai_voice_enrollments'
export function loadEnrollments() {
try {
const raw = localStorage.getItem(ENROLL_KEY)
if (!raw) return []
const p = JSON.parse(raw)
return Array.isArray(p) ? p : []
} catch (_) { return [] }
}
export function saveEnrollments(list) {
try { localStorage.setItem(ENROLL_KEY, JSON.stringify(list.slice(0, 50))) } catch (_) { /* quota */ }
}
// Only registration metadata belongs in the browser's management list.
export function rememberEnrollment({ id, name, registered_at }) {
saveEnrollments([{ id, name, registeredAt: registered_at, labels: {} }, ...loadEnrollments()])
}
addons/studio/ui/test/diarization.test.mjs
+65
View File
@@ -0,0 +1,65 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { axisTicks, clock, hasText, runLength, speakerRows, toRttm, toSrt } from '../src/utils/diarization.js'
const result = {
speakers: [{ id: 'SPEAKER_00', label: '0' }, { id: 'SPEAKER_01', label: '1', name: 'Ada' }],
segments: [
{ id: 0, speaker: 'SPEAKER_00', label: '0', start: 0, end: 6, text: 'Morning everyone.' },
{ id: 1, speaker: 'SPEAKER_01', label: '1', start: 6.5, end: 8.5, text: ' Hello. ' },
{ id: 2, speaker: 'SPEAKER_00', label: '0', start: 9, end: 11 },
],
}
test('clock reads minutes and seconds, hours only when needed', () => {
assert.equal(clock(0), '0:00')
assert.equal(clock(74.6), '1:15')
assert.equal(clock(3725), '1:02:05')
assert.equal(clock('x'), '0:00')
})
test('speakerRows sums talk time per speaker in order of first speech', () => {
const rows = speakerRows(result)
assert.deepEqual(rows.map(r => r.label), ['0', '1'])
assert.equal(rows[0].seconds, 8)
assert.equal(rows[1].seconds, 2)
assert.equal(rows[1].name, 'Ada')
assert.ok(Math.abs(rows[0].share - 0.8) < 1e-9)
})
test('speakerRows keeps a speaker that has no segment', () => {
const rows = speakerRows({ speakers: [{ id: 'S', label: '4' }], segments: [] })
assert.equal(rows.length, 1)
assert.equal(rows[0].share, 0)
})
test('runLength is the last end, zero when empty', () => {
assert.equal(runLength(result), 11)
assert.equal(runLength({}), 0)
})
test('axisTicks steps over round numbers and spans the run', () => {
const ticks = axisTicks(168)
assert.equal(ticks[0].at, 0)
assert.ok(ticks.length >= 4 && ticks.length <= 8)
assert.ok(ticks.every(t => t.pct >= 0 && t.pct <= 100))
assert.deepEqual(axisTicks(0), [])
})
test('toRttm writes one line per segment and uses a known name', () => {
const lines = toRttm(result, 'team call.wav').trim().split('\n')
assert.equal(lines.length, 3)
assert.equal(lines[0], 'SPEAKER team_call 1 0.000 6.000 <NA> <NA> SPEAKER_00 <NA> <NA>')
assert.match(lines[1], /Ada/)
})
test('toSrt keeps only segments with text and numbers them', () => {
const srt = toSrt(result)
assert.match(srt, /^1\n00:00:00,000 --> 00:00:06,000\nSPEAKER_00: Morning everyone\./)
assert.match(srt, /\n2\n00:00:06,500 --> 00:00:08,500\nAda: Hello\./)
assert.equal(srt.includes('\n3\n'), false)
assert.equal(hasText(result), true)
assert.equal(hasText({ segments: [{ start: 0, end: 1 }] }), false)
assert.equal(toSrt({ segments: [] }), '')
})
addons/studio/ui/test/parity.test.mjs
+53
View File
@@ -0,0 +1,53 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { hasKey, hostUi, iconLiterals, iconNames, literalKeys, sourceFiles, spriteNames, text, unresolvedImports } from '../../../internal/host-compat/test/parity.mjs'
const here = dirname(fileURLToPath(import.meta.url))
const src = join(here, '../src')
const files = sourceFiles(src)
const LANGUAGES = ['en', 'it', 'es', 'de', 'zh-CN', 'id', 'ko', 'pt-BR']
test('the media locale files are byte copies of the host ones', () => {
for (const lng of LANGUAGES) {
assert.equal(
readFileSync(join(src, `i18n/media.${lng}.json`), 'utf8'),
readFileSync(join(hostUi, `public/locales/${lng}/media.json`), 'utf8'),
lng,
)
}
})
test('every literal key the pages read is in the English media locale', () => {
const en = JSON.parse(readFileSync(join(src, 'i18n/media.en.json'), 'utf8'))
const keys = literalKeys(text(files))
assert.ok(keys.size > 150, `found only ${keys.size} literal keys`)
const missing = [...keys].filter((k) => !hasKey(en, k))
assert.deepEqual(missing, [])
})
test('every icon the pages name is one the SDK allows', async () => {
const allowed = await iconNames()
const sprite = spriteNames()
const used = iconLiterals(text(files))
assert.ok(used.size > 20, `found only ${used.size} icons`)
// Each is also a real sprite icon, so none draws the neutral glyph.
assert.deepEqual([...used].filter((n) => !sprite.has(n) || !allowed.has(n)), [])
})
test('the source imports only its own files, the compat stand-ins and the modules they answer', () => {
assert.deepEqual(unresolvedImports(src), [])
})
test('the source holds no direct network call: the pages go through the stand-in API client', () => {
const direct = []
for (const f of files) {
if (f.endsWith('test.mjs')) continue
for (const line of readFileSync(f, 'utf8').split('\n')) {
if (/\bfetch\(/.test(line) && !/^\s*\/\//.test(line)) direct.push(`${f.slice(src.length)}: ${line.trim()}`)
}
}
assert.deepEqual(direct, [])
})
addons/studio/ui/test/render.test.mjs
+61
View File
@@ -0,0 +1,61 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { build } from '../build.mjs'
import { React, renderToString, makeSdk } from '../../../internal/host-compat/test/host.mjs'
const here = dirname(fileURLToPath(import.meta.url))
async function load(overrides) {
const dir = await build()
const mod = await import(pathToFileURL(join(dir, 'ui/studio.js')).href + `?t=${Date.now()}`)
const host = await makeSdk({ addonId: 'studio', pages: ['studio'], overrides })
return { mod, ...host }
}
test('render: createAddon returns a page component and dispose', async () => {
const { mod, sdk, controller } = await load()
const addon = await mod.createAddon(sdk)
assert.equal(typeof addon.Component, 'function')
assert.equal(typeof addon.dispose, 'function')
addon.dispose()
controller.abort('done')
})
test('render: an unknown page draws nothing', async () => {
const { mod, sdk } = await load()
const addon = await mod.createAddon(sdk)
assert.equal(renderToString(React.createElement(addon.Component, { page: 'nope', path: '', search: '' })), '')
})
test('render: the overview draws the tab strip with its words from the media locale', async () => {
const { mod, sdk } = await load()
const addon = await mod.createAddon(sdk)
const html = renderToString(React.createElement(addon.Component, { page: 'studio', path: '', search: '' }))
assert.match(html, /class="studio-tabs"/)
assert.match(html, /data-tab="images"/)
assert.match(html, /data-tab="overview"/)
assert.doesNotMatch(html, /studio\.tabs\./, 'a key was shown instead of its words')
})
test('render: every tab draws without throwing', async () => {
const { mod, sdk } = await load()
const addon = await mod.createAddon(sdk)
for (const tab of ['images', 'video', 'threed', 'tts', 'sound', 'transform', 'diarization']) {
const html = renderToString(React.createElement(addon.Component, { page: 'studio', path: tab, search: '' }))
assert.match(html, new RegExp(`studio-tab-active[^>]*|data-tab="${tab}"`), tab)
assert.doesNotMatch(html, /[a-z]+\.[a-z]+\.[a-zA-Z.]+<\/(span|h1|p|button|label)>/, `${tab}: a key was shown instead of its words`)
}
})
test('render: the manifest declares one page, the main page and SDK 2', () => {
const m = JSON.parse(readFileSync(join(here, '../../manifest.json'), 'utf8'))
assert.equal(m.id, 'studio')
assert.deepEqual(m.capabilities.ui.pages, ['studio'])
assert.deepEqual(m.capabilities.ui.navigation, ['studio'])
assert.equal(m.capabilities.ui.mainPage, 'studio')
assert.deepEqual(m.capabilities.ui.sdk, { min: 2, max: 2 })
assert.deepEqual(m.capabilities.tools ?? [], [])
})
addons/studio/ui/test/run.mjs
+23
View File
@@ -0,0 +1,23 @@
// Runs the node unit tests of the Studio UI package and prints one JSON line
// per test, the same shape as the host's runner (core/http/react-ui/tests/unit):
// {"case":"unit: <name>","status":"passed"|"failed"|"skipped"}
import { run } from 'node:test'
import { readdirSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
const here = dirname(fileURLToPath(import.meta.url))
const names = readdirSync(here).filter(f => f.endsWith('.test.mjs')).sort()
let failed = false
const stream = run({ files: names.map(f => join(here, f)), concurrency: 1 })
stream.on('test:pass', ({ name, skip, todo, nesting }) => {
if (nesting > 0) return
console.log(JSON.stringify({ case: `unit: ${name}`, status: skip || todo ? 'skipped' : 'passed' }))
})
stream.on('test:fail', ({ name, nesting, details }) => {
if (nesting > 0) return
failed = true
console.log(JSON.stringify({ case: `unit: ${name}`, status: 'failed', error: String(details && details.error && details.error.message || 'failed').slice(0, 400) }))
})
stream.on('end', () => { process.exit(failed ? 1 : 0) })
stream.resume()
addons/studio/ui/test/studioWork.test.mjs
+263
View File
@@ -0,0 +1,263 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import {
MAX_FAVOURITES, TYPE_ORDER, canTake, collectWork, columnCount, countByType, dealColumns, filterWork,
groupWork, handoffPath, layoutLineage, neighbour, projectOf, pruneIds, readHandoff, sourceUrlFor,
suggestNext, suggestType, toggleId, waveBars,
} from '../src/utils/studioWork.js'
import {
FAVOURITES_KEY, MEDIA_STORAGE_KEYS, clearAllMediaHistory,
} from '../src/utils/mediaHistoryStore.js'
const entry = (id, createdAt, extra = {}) => ({
id, createdAt, prompt: `prompt ${id}`, model: 'm', params: {}, results: [{ url: `/generated/${id}.png` }], ...extra,
})
// A harbour: a picture and a second take of it, a clip animated from the first,
// and an unrelated sound.
function harbour() {
const media = {
image: [entry('i1', 100, { params: { size: '1216x832' } }), entry('i2', 110, { parentId: 'i1', edge: 'take' })],
video: [entry('v1', 200, { parentId: 'i1', edge: 'animate' })],
tts: [entry('t1', 300)],
sound: [], 'audio-transform': [], diarization: [],
}
return collectWork(media, [], { favourites: ['i2'] })
}
test('suggestType picks a type from the words and stays quiet otherwise', () => {
assert.equal(suggestType('slow dolly through a misty forest, drone shot'), 'video')
assert.equal(suggestType('a 3d mesh of a ceramic bowl'), 'threed')
assert.equal(suggestType('read this aloud in a calm voice'), 'tts')
assert.equal(suggestType('lo-fi piano with rain'), 'sound')
assert.equal(suggestType('who spoke in this meeting'), 'diarization')
assert.equal(suggestType('please clean up this recording'), 'transform')
assert.equal(suggestType('a poster of a lighthouse'), 'images')
assert.equal(suggestType('hello there'), null)
assert.equal(suggestType('video'), null, 'too short to suggest from')
assert.equal(suggestType(''), null)
})
test('narrow cases win over broad ones', () => {
// "picture" is an image word, but "video" is the more specific ask.
assert.equal(suggestType('a video of a picture frame falling'), 'video')
assert.equal(suggestType('who spoke in this video clip'), 'diarization')
})
test('collectWork normalises every store, newest first, with favourites', () => {
const items = harbour()
assert.deepEqual(items.map(i => i.id), ['t1', 'v1', 'i2', 'i1'])
assert.equal(items.find(i => i.id === 'i2').favourite, true)
assert.equal(items.find(i => i.id === 'i1').favourite, false)
assert.equal(items.find(i => i.id === 'v1').type, 'video')
assert.equal(items.find(i => i.id === 'v1').url, '/generated/v1.png')
})
test('a 3D entry takes its title from the label, else the file name, and keeps its thumbnail', () => {
const items = collectWork({}, [
{ id: 'm1', createdAt: 5, name: 'a.glb', label: 'Ceramic bowl', inputThumb: 'data:image/jpeg;base64,AA', model: 'trellis' },
{ id: 'm2', createdAt: 4, name: 'b.glb' },
])
assert.equal(items[0].title, 'Ceramic bowl')
assert.equal(items[0].thumb, 'data:image/jpeg;base64,AA')
assert.equal(items[1].title, 'b.glb')
})
test('a transform entry keeps its input apart from its output', () => {
const [item] = collectWork({
'audio-transform': [entry('x1', 1, { results: [{ kind: 'output', url: '/o.wav' }, { kind: 'input', url: '/i.wav' }] })],
}, [])
assert.equal(item.url, '/o.wav')
assert.equal(item.inputUrl, '/i.wav')
assert.deepEqual(item.urls, ['/o.wav'])
})
test('counts are per result and filters match the counts', () => {
const items = harbour()
const counts = countByType(items)
assert.equal(counts.all, 4)
assert.equal(counts.images, 2)
assert.equal(counts.video, 1)
assert.equal(counts.favourites, 1)
for (const key of TYPE_ORDER) assert.equal(filterWork(items, key).length, counts[key])
assert.equal(filterWork(items, 'favourites').length, counts.favourites)
assert.equal(filterWork(items, 'all').length, counts.all)
})
test('related results stack into one project and the rest stay single', () => {
const tiles = groupWork(harbour())
assert.equal(tiles.length, 2)
const project = tiles.find(t => t.kind === 'project')
assert.equal(project.items.length, 3)
assert.equal(project.id, 'i1', 'the project is named by its root result')
assert.equal(project.cover.id, 'i2', 'the newest picture is the cover')
assert.equal(tiles.find(t => t.kind === 'single').item.id, 't1')
assert.equal(tiles[0].kind, 'single', 'newest activity first')
})
test('a link to a result that was deleted leaves the child standing alone', () => {
const items = collectWork({ image: [entry('a', 1, { parentId: 'gone', edge: 'take' })], video: [], tts: [], sound: [], 'audio-transform': [], diarization: [] }, [])
assert.equal(groupWork(items).length, 1)
assert.equal(groupWork(items)[0].kind, 'single')
})
test('a loop in stored links does not hang or stack', () => {
const media = { image: [entry('a', 1, { parentId: 'b' }), entry('b', 2, { parentId: 'a' })], video: [], tts: [], sound: [], 'audio-transform': [], diarization: [] }
const items = collectWork(media, [])
const tiles = groupWork(items)
assert.ok(tiles.length >= 1)
const layout = layoutLineage(projectOf(items, 'a'))
assert.ok(layout.nodes.every(n => Number.isFinite(n.x) && Number.isFinite(n.y)))
})
test('projectOf returns the whole project from any member, oldest first', () => {
const items = harbour()
assert.deepEqual(projectOf(items, 'v1').map(i => i.id), ['i1', 'i2', 'v1'])
assert.deepEqual(projectOf(items, 't1').map(i => i.id), ['t1'])
assert.equal(projectOf(items, 'nope'), null)
})
test('the lineage lays out a prompt, two takes sharing it, and a clip from the first', () => {
const members = projectOf(harbour(), 'v1')
const layout = layoutLineage(members, { selectedId: 'v1' })
const byId = Object.fromEntries(layout.nodes.map(n => [n.id, n]))
assert.ok(byId['src:i1'], 'one prompt node')
assert.equal(layout.nodes.filter(n => n.kind === 'source').length, 1)
const edge = (from, to) => layout.edges.find(e => e.from === from && e.to === to)
assert.equal(edge('src:i1', 'res:i1').label, null)
assert.equal(edge('src:i1', 'res:i2').label, 'take', 'a take hangs off the shared prompt')
assert.equal(edge('res:i1', 'res:v1').label, 'animate')
assert.ok(byId['res:v1'].x > byId['res:i1'].x && byId['res:i1'].x > byId['src:i1'].x, 'columns follow depth')
// The selected path, root to selection, is the heavy one.
assert.ok(edge('src:i1', 'res:i1').chain && edge('res:i1', 'res:v1').chain)
assert.ok(!edge('src:i1', 'res:i2').chain)
assert.deepEqual([...layout.chain].sort(), ['res:i1', 'res:v1', 'src:i1'])
})
test('a file start gets a file node, a prompt start gets the prompt text', () => {
const media = {
image: [entry('i1', 1)], video: [], tts: [], sound: [], diarization: [],
'audio-transform': [entry('x1', 2, { prompt: 'audio: a.wav' })],
}
const layout = layoutLineage(collectWork(media, []).filter(i => i.id === 'x1'))
const source = layout.nodes.find(n => n.kind === 'source')
assert.equal(source.input, 'file')
assert.equal(source.title, '')
})
test('a ghost node hangs off the selection and the board grows to hold it', () => {
const members = projectOf(harbour(), 'v1')
const plain = layoutLineage(members, { selectedId: 'v1' })
const ghosted = layoutLineage(members, { selectedId: 'v1', ghost: { id: 'ghost:v1', from: 'res:v1', to: 'sound', edge: 'soundtrack' } })
const ghost = ghosted.nodes.find(n => n.kind === 'ghost')
assert.ok(ghost)
assert.ok(ghost.x > ghosted.nodes.find(n => n.id === 'res:v1').x)
assert.ok(ghosted.width > plain.width)
assert.ok(ghosted.edges.find(e => e.ghost).chain)
})
test('arrow keys walk left to the source result, right to a child, and down a column', () => {
const layout = layoutLineage(projectOf(harbour(), 'v1'), { selectedId: 'v1' })
assert.equal(neighbour(layout, 'v1', 'left'), 'i1')
assert.equal(neighbour(layout, 'i1', 'right'), 'v1')
assert.equal(neighbour(layout, 'i1', 'down'), 'i2')
assert.equal(neighbour(layout, 'i2', 'up'), 'i1')
assert.equal(neighbour(layout, 'i1', 'left'), 'i1', 'the first result has nothing to its left but a prompt')
})
test('the suggested next step is the first supported one with a model, else the first supported', () => {
assert.equal(suggestNext('images', new Set(['images', 'threed'])).to, 'threed')
assert.equal(suggestNext('images', new Set(['images', 'video'])).to, 'video')
assert.equal(suggestNext('images', new Set(['images'])).to, 'images', 'a variation needs only the model already installed')
assert.equal(suggestNext('images', new Set()).to, 'video', 'none installed: the first, marked as missing by the caller')
assert.equal(suggestNext('video', new Set(['sound'])), null, 'the Sound page cannot start from a clip')
assert.equal(suggestNext('diarization', new Set()), null)
})
test('run as a new take needs the original input', () => {
const items = Object.fromEntries(harbour().map(i => [i.id, i]))
assert.equal(canTake(items.i1).ok, true)
assert.equal(canTake({ type: 'threed' }).ok, false)
assert.equal(canTake({ type: 'diarization' }).ok, false)
assert.equal(canTake({ type: 'transform', inputUrl: '' }).ok, false)
assert.equal(canTake({ type: 'transform', inputUrl: '/i.wav' }).ok, true)
})
test('hand-off paths carry what a workspace accepts and read back the same', () => {
const path = handoffPath('images', { prompt: ' a brass orrery ', model: 'flux', size: '768x768', count: 3, from: 'i1', edge: 'variation' })
assert.match(path, /^\/app\/studio\/images\?/)
const got = readHandoff(new URL(path, 'http://x').searchParams)
assert.deepEqual(got, { prompt: 'a brass orrery', model: 'flux', size: '768x768', count: 3, from: 'i1', edge: 'variation' })
assert.equal(handoffPath('tts', {}), '/app/studio/tts')
assert.equal(handoffPath('images', { count: 1 }), '/app/studio/images', 'a count of one is the default')
assert.deepEqual(readHandoff(new URLSearchParams('n=abc')).count, 0)
})
test('a take of a transform starts from its input, anything else from its output', () => {
const item = { url: '/o.wav', inputUrl: '/i.wav' }
assert.equal(sourceUrlFor(item, 'take'), '/i.wav')
assert.equal(sourceUrlFor(item, 'diarize'), '/o.wav')
assert.equal(sourceUrlFor(null, 'take'), '')
})
test('favourites toggle, stay bounded, and drop ids that no longer exist', () => {
assert.deepEqual(toggleId([], 'a'), ['a'])
assert.deepEqual(toggleId(['a', 'b'], 'a'), ['b'])
let ids = []
for (let i = 0; i < MAX_FAVOURITES + 25; i++) ids = toggleId(ids, `id${i}`)
assert.equal(ids.length, MAX_FAVOURITES)
assert.equal(ids[ids.length - 1], `id${MAX_FAVOURITES + 24}`, 'the newest is kept')
assert.deepEqual(pruneIds(['i1', 'zzz'], harbour()), ['i1'])
})
test('clearing history removes every list and the favourites, and nothing else', () => {
const store = new Map([['unrelated', 'keep'], [FAVOURITES_KEY, '["a"]']])
for (const key of Object.values(MEDIA_STORAGE_KEYS)) store.set(key, '[{"id":"a"}]')
globalThis.localStorage = {
getItem: k => (store.has(k) ? store.get(k) : null),
setItem: (k, v) => store.set(k, v),
removeItem: k => store.delete(k),
}
const removed = clearAllMediaHistory()
assert.equal(removed, Object.keys(MEDIA_STORAGE_KEYS).length + 1)
assert.deepEqual([...store.keys()], ['unrelated'])
assert.equal(clearAllMediaHistory(), 0, 'clearing again removes nothing')
delete globalThis.localStorage
})
test('every workspace has a history list, including diarization', () => {
assert.ok(MEDIA_STORAGE_KEYS.diarization)
for (const key of ['image', 'video', 'tts', 'sound', 'audio-transform']) assert.ok(MEDIA_STORAGE_KEYS[key])
})
test('the masonry balances columns and follows the width', () => {
assert.equal(columnCount(0), 1)
assert.equal(columnCount(320), 1)
assert.equal(columnCount(700), 2)
assert.equal(columnCount(1000), 3)
assert.equal(columnCount(5000), 4)
const tiles = groupWork(harbour()).concat(
...[1, 2, 3, 4, 5].map(n => groupWork(collectWork({ image: [entry(`z${n}`, n)], video: [], tts: [], sound: [], 'audio-transform': [], diarization: [] }, []))),
)
const cols = dealColumns(tiles, 3)
assert.equal(cols.length, 3)
assert.equal(cols.flat().length, tiles.length, 'every tile is dealt once')
assert.ok(cols.every(c => c.length > 0))
})
test('a waveform placeholder is repeatable and bounded', () => {
const a = waveBars('t1')
assert.deepEqual(a, waveBars('t1'))
assert.notDeepEqual(a, waveBars('t2'))
assert.ok(a.every(h => h > 0 && h <= 1))
assert.equal(waveBars('x', 10).length, 10)
})
test('a 3D result is titled by the label, else the motion prompt, else the file name', () => {
const base = { id: 'm1', createdAt: 1, model: 'trellis', params: {}, name: 'out.glb' }
const title = (extra) => collectWork({}, [{ ...base, ...extra }])[0].title
assert.equal(title({ label: 'A vase' }), 'A vase')
assert.equal(title({ inputs: { prompt: { type: 'text', data: 'Walk forward and wave' } } }), 'Walk forward and wave')
assert.equal(title({}), 'out.glb')
})
addons/studio/ui/test/upscale.test.mjs
+45
View File
@@ -0,0 +1,45 @@
// SPDX-License-Identifier: MIT
import test from 'node:test'
import assert from 'node:assert/strict'
import '../../../internal/host-compat/test/register.mjs'
// upscale.js imports a module that lives in the compat directory.
const { upscaleModels, sourceImageFile } = await import('../src/utils/upscale.js')
import { toWorkItem, EDGE_KINDS } from '../src/utils/studioWork.js'
test('only enabled upscalers with positive finite scale are offered', () => {
assert.deepEqual(upscaleModels([
{ id: 'yes', capabilities: ['FLAG_UPSCALE'], upscaleScale: 4 },
{ id: 'image', capabilities: ['FLAG_IMAGE'], upscaleScale: 4 },
...[0, -1, undefined, Infinity].map(upscaleScale => ({ capabilities: ['FLAG_UPSCALE'], upscaleScale })),
{ capabilities: ['FLAG_UPSCALE'], upscaleScale: 2, disabled: true },
]), [{ id: 'yes', upscaleScale: 4 }])
})
test('URL source is same-origin, checked and named', async () => {
let called = false
const fetcher = async (url, options) => {
called = true
assert.equal(url, 'https://local.test/generated/source.png')
assert.equal(options.credentials, 'same-origin')
return new Response('png', { headers: { 'Content-Type': 'image/png' } })
}
const file = await sourceImageFile('/generated/source.png', 'https://local.test/app/studio', fetcher)
assert.equal(file.name, 'source.png')
assert.equal(file.type, 'image/png')
called = false
await assert.rejects(sourceImageFile('https://other.test/a.png', 'https://local.test/', fetcher), /origin/)
assert.equal(called, false)
for (const response of [new Response('bad', { status: 500 }), new Response('html', { headers: { 'Content-Type': 'text/html' } })]) {
await assert.rejects(sourceImageFile('/a', 'https://local.test/', async () => response))
}
await assert.rejects(sourceImageFile('/a', 'https://local.test/', async () => { throw new Error('offline') }))
})
test('upscale history normalizes as a child image', () => {
assert.ok(EDGE_KINDS.includes('upscale'))
const item = toWorkItem('images', { id: 'child', parentId: 'source', edge: 'upscale', results: [{ url: '/out.png' }] })
assert.equal(item.type, 'images')
assert.equal(item.parentId, 'source')
assert.equal(item.edge, 'upscale')
})