From 4906cbad0467d5f31b8a148648a03686c2950309 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 24 Apr 2026 08:50:34 +0200 Subject: [PATCH] feat: add biometrics UI (#9524) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(react-ui): add Face & Voice Recognition pages Expose the face and voice biometrics endpoints (/v1/face/*, /v1/voice/*) through the React UI. Each page has four tabs driving the six endpoints per modality: Analyze (demographics with bounding boxes / waveform segments), Compare (verify with a match gauge and live threshold slider), Enrollment (register / identify / forget with a top-K matches view), Embedding (raw vector inspector with sparkline + copy). MediaInput supports file upload plus live capture: webcam snap-to-canvas for face, MediaRecorder -> AudioContext -> 16-bit PCM mono WAV transcode for voice (libsndfile on the backend only handles WAV/FLAC/OGG natively). Sidebar gets a new Biometrics section feature-gated on face_recognition / voice_recognition; routes are wrapped in . No new dependencies -- Font Awesome icons picked from the Free set. Assisted-by: Claude:Opus 4.7 * fix(localai): accept data URI prefixes with codec/charset params Browser MediaRecorder produces data URIs like data:audio/webm;codecs=opus;base64,... so the pre-';base64,' section can carry multiple parameter segments. The `^data:([^;]+);base64,` regex in pkg/utils/base64.go and core/http/endpoints/localai/audio.go only matched exactly one segment, so recordings straight from the React UI's live-capture tab failed the strip and then tripped the base64 decoder on the leading 'data:' literal, surfacing as "invalid audio base64: illegal base64 data at input byte 4" Widened both regexes to `^data:[^,]+?;base64,` so any number of ';param=value' segments between the mime type and ';base64,' are tolerated. Added a regression test covering the MediaRecorder shape. Assisted-by: Claude:Opus 4.7 * fix(insightface): scope pack ONNX loading to known manifests LocalAI's gallery extracts buffalo_* zips flat into the models directory, which inevitably mixes with ONNX files from other backends (opencv face engine, MiniFASNet antispoof, WeSpeaker voice embedding) and older buffalo pack installs. Feeding those foreign files into insightface's model_zoo.get_model() blows up inside the router -- it assumes a 4-D NCHW input and indexes `input_shape[2]` on tensors that aren't shaped like a face model, raising IndexError mid-load and leaving the backend unusable. The router's dispatch isn't amenable to per-file try/except alone (first-file-wins picks det_10g.onnx from buffalo_l even when the user asked for buffalo_sc -- alphabetical order happens to favour the wrong pack). Instead, ship an explicit manifest of the upstream v0.7 pack contents and scope the glob to that when the requested pack is known. The manifest is small and stable; future packs can be added alongside or fall through to the tolerance loop, which also swallows any remaining IndexError / ValueError from foreign files with a clear `[insightface] skipped` stderr line for diagnostics. Assisted-by: Claude:Opus 4.7 * fix(speaker-recognition): extract FBank features for rank-3 ONNX encoders Pre-exported speaker-encoder ONNX graphs come in two shapes: rank-2 [batch, samples] -- some 3D-Speaker exports, take raw waveform directly. rank-3 [batch, frames, n_mels] -- WeSpeaker and most Kaldi- lineage encoders, expect pre-computed Kaldi FBank. OnnxDirectEngine unconditionally fed `audio.reshape(1, -1)` -- correct for rank-2, IndexError-on-input_shape[3] on rank-3, which surfaced to the UI as "Invalid rank for input: feats Got: 2 Expected: 3" Detect the input rank at session init and run Kaldi FBank (80-dim, 25ms/10ms frames, dither=0.0, per-utterance CMN) before the forward pass when rank>=3. All knobs are configurable via backend options for encoders that deviate from defaults. torchaudio.compliance.kaldi is already in the backend's requirements (SpeechBrain pulls torchaudio in), so no new dependency. Assisted-by: Claude:Opus 4.7 * fix(biometrics): isolate face and voice vector stores Face (ArcFace, 512-D) and voice (ECAPA-TDNN 192-D / WeSpeaker 256-D) biometric embeddings were colliding inside a single in-memory local-store instance. Enrolling one after the other failed with "Try to add key with length N when existing length is M" because local-store correctly refuses to mix dimensions in one keyspace. The registries were constructed with `storeName=""`, which in StoreBackend() is just a WithModel() call. But ModelLoader's cache is keyed on `modelID`, not `model` -- so both registries collapsed to the same `modelID=""` slot and reused the same backend process despite looking isolated on paper. Three complementary fixes: 1. application.go -- give each registry a distinct default namespace ("localai-face-biometrics" / "localai-voice-biometrics"). The comment claimed isolation, now it's actually enforced. 2. stores.go -- pass the storeName as both WithModelID and WithModel so the ModelLoader cache key separates namespaces and the loader spawns distinct processes. 3. local-store/store.go -- drop the Load() `opts.Model != ""` guard. It was there to prevent generic model-loading loops from picking up local-store by accident, but that auto-load path is being retired; the guard now just blocks legitimate namespace isolation. opts.Model is treated as a tag; the per-tuple process isolation upstream handles discrimination. Assisted-by: Claude:Opus 4.7 * fix(gallery): stale-file cleanup and upgrade-tmp directory safety Two related robustness fixes for backend install/upgrade: pkg/downloader/uri.go OCI downloads passed through if filepath.Ext(filePath) != "" ... filePath = filepath.Dir(filePath) which was intended to redirect file-shaped download targets into their parent directory for OCI extraction. The heuristic misfires on directory-shaped paths with a dot-suffix -- gallery.UpgradeBackend uses tmpPath = "/.upgrade-tmp" and Go's filepath.Ext treats ".upgrade-tmp" as an extension. The rewrite landed the extraction at "/", which then **overwrote the real install** (backends//) with a flat-layout file and left a stray run.sh at the top level. The tmp dir itself stayed empty, so the validation step that checked "/run.sh" predictably failed with "upgrade validation failed: run.sh not found in new backend" Every manual upgrade silently corrupted the backends tree this way. Guard the rewrite behind "target isn't already an existing directory" -- InstallBackend / UpgradeBackend both pre-create the target as a directory, so they get the correct behaviour; existing file-path callers with a genuine dot-extension still get the parent redirect. core/gallery/backends.go InstallBackend's MkdirAll returned ENOTDIR when something at the target path was already a file (legacy dev builds dropped golang backend binaries directly at `/` instead of nesting them under their own subdir). That permanently blocked reinstall and upgrade for anyone carrying that state, since every retry hit the same error. Detect a pre-existing non-directory, warn, and remove it before the MkdirAll so the fresh install can write the correct nested layout with metadata.json + run.sh. Assisted-by: Claude:Opus 4.7 * fix(galleryop): refresh upgrade cache after backend ops UpgradeChecker caches the last upgrade-check result and only refreshes on the 6-hour tick or after an auto-upgrade cycle. Manual upgrades (POST /api/backends/upgrade/:name) go through the async galleryop worker, which completes the upgrade correctly but never tells UpgradeChecker to re-check -- so /api/backends/upgrades continued to list a just-upgraded backend as upgradeable, indistinguishable from a failed upgrade, for up to six hours. Add an optional `OnBackendOpCompleted func()` hook on GalleryService that fires after every successful install / upgrade / delete on the backend channel (async, so a slow callback doesn't stall the queue). startup.go wires it to UpgradeChecker.TriggerCheck after both services exist. Result: the upgrade banner clears within milliseconds of the worker finishing. Assisted-by: Claude:Opus 4.7 * build: prepend GOPATH/bin to PATH for protogen-go install-go-tools runs `go install` for protoc-gen-go and protoc-gen-go-grpc, which writes them into `go env GOPATH`/bin. That directory isn't on every dev's PATH, and protoc resolves its code-gen plugins via PATH, so the immediately-following protoc invocation fails with "protoc-gen-go: program not found" which in turn blocks `make build` and any `make backends/%` target that depends on build. Prepend `go env GOPATH`/bin to PATH for the protoc invocation so the freshly-installed plugins are found without requiring a shell-profile change. Assisted-by: Claude:Opus 4.7 * refactor(ui-api): non-blocking backend upgrade handler with opcache POST /api/backends/upgrade/:name used to send the ManagementOp directly onto the unbuffered BackendGalleryChannel, which blocked the HTTP request whenever the galleryop worker was busy with a prior operation. The op also didn't show up in /api/operations, so the Backends UI couldn't reflect upgrade progress on the affected row. Register the op in opcache immediately, wrap it in a cancellable context, store the cancellation function on the GalleryService, and push onto the channel from a goroutine so the handler returns right away. Response gains a `jobID` field and a `message` string so clients have a consistent handle regardless of whether the op is queued or running. Pairs with the OnBackendOpCompleted hook added in the galleryop commit — together the UI sees the upgrade start, watches progress via /api/operations, and drops the "upgradeable" flag the moment the worker finishes. Assisted-by: Claude:Opus 4.7 --- Makefile | 8 +- backend/go/local-store/store.go | 14 +- backend/python/insightface/engines.py | 58 +- backend/python/speaker-recognition/engines.py | 45 +- core/application/application.go | 20 +- core/application/startup.go | 6 + core/backend/stores.go | 9 + core/gallery/backends.go | 14 + core/http/endpoints/localai/audio.go | 8 +- core/http/react-ui/src/App.css | 1260 +++++++++++++++++ core/http/react-ui/src/components/Sidebar.jsx | 12 + .../biometrics/BoundingBoxCanvas.jsx | 63 + .../biometrics/DistributionBars.jsx | 33 + .../biometrics/EmbeddingInspector.jsx | 89 ++ .../components/biometrics/EnrollmentList.jsx | 65 + .../src/components/biometrics/MatchGauge.jsx | 46 + .../src/components/biometrics/MediaInput.jsx | 179 +++ .../src/components/biometrics/TabSwitch.jsx | 22 + .../components/biometrics/WaveformStrip.jsx | 99 ++ .../react-ui/src/hooks/useMediaCapture.js | 205 +++ .../react-ui/src/pages/FaceRecognition.jsx | 602 ++++++++ .../react-ui/src/pages/VoiceRecognition.jsx | 543 +++++++ core/http/react-ui/src/router.jsx | 6 + core/http/react-ui/src/utils/api.js | 20 + core/http/react-ui/src/utils/config.js | 17 + core/http/routes/ui_api.go | 28 +- core/services/galleryop/service.go | 12 + pkg/downloader/uri.go | 17 +- pkg/utils/base64.go | 8 +- pkg/utils/base64_test.go | 9 + 30 files changed, 3495 insertions(+), 22 deletions(-) create mode 100644 core/http/react-ui/src/components/biometrics/BoundingBoxCanvas.jsx create mode 100644 core/http/react-ui/src/components/biometrics/DistributionBars.jsx create mode 100644 core/http/react-ui/src/components/biometrics/EmbeddingInspector.jsx create mode 100644 core/http/react-ui/src/components/biometrics/EnrollmentList.jsx create mode 100644 core/http/react-ui/src/components/biometrics/MatchGauge.jsx create mode 100644 core/http/react-ui/src/components/biometrics/MediaInput.jsx create mode 100644 core/http/react-ui/src/components/biometrics/TabSwitch.jsx create mode 100644 core/http/react-ui/src/components/biometrics/WaveformStrip.jsx create mode 100644 core/http/react-ui/src/hooks/useMediaCapture.js create mode 100644 core/http/react-ui/src/pages/FaceRecognition.jsx create mode 100644 core/http/react-ui/src/pages/VoiceRecognition.jsx diff --git a/Makefile b/Makefile index 578d119a7..43f66f5cb 100644 --- a/Makefile +++ b/Makefile @@ -394,7 +394,13 @@ protoc: .PHONY: protogen-go protogen-go: protoc install-go-tools mkdir -p pkg/grpc/proto - ./protoc --experimental_allow_proto3_optional -Ibackend/ --go_out=pkg/grpc/proto/ --go_opt=paths=source_relative --go-grpc_out=pkg/grpc/proto/ --go-grpc_opt=paths=source_relative \ + # install-go-tools writes protoc-gen-go and protoc-gen-go-grpc into + # $(shell go env GOPATH)/bin, which isn't on every dev's PATH. protoc + # resolves its code-gen plugins via PATH, so without this prefix the + # generate step fails with "protoc-gen-go: program not found". Prepend + # GOPATH/bin so the freshly-installed plugins win without requiring a + # shell-profile change. + PATH="$$(go env GOPATH)/bin:$$PATH" ./protoc --experimental_allow_proto3_optional -Ibackend/ --go_out=pkg/grpc/proto/ --go_opt=paths=source_relative --go-grpc_out=pkg/grpc/proto/ --go-grpc_opt=paths=source_relative \ backend/backend.proto core/config/inference_defaults.json: ## Fetch inference defaults from unsloth (only if missing) diff --git a/backend/go/local-store/store.go b/backend/go/local-store/store.go index b48c2e919..e2ad54098 100644 --- a/backend/go/local-store/store.go +++ b/backend/go/local-store/store.go @@ -4,7 +4,6 @@ package main // It is meant to be used by the main executable that is the server for the specific backend type (falcon, gpt3, etc) import ( "container/heap" - "errors" "fmt" "math" "slices" @@ -100,9 +99,16 @@ func sortIntoKeySlicese(keys []*pb.StoresKey) [][]float32 { } func (s *Store) Load(opts *pb.ModelOptions) error { - if opts.Model != "" { - return errors.New("not implemented") - } + // local-store is an in-memory vector store with no on-disk artefact to + // load — opts.Model is just a namespace identifier. The old `!= ""` guard + // rejected any non-empty model name with "not implemented", which broke + // callers that pass a namespace to isolate embedding spaces (face vs. + // voice biometrics both go through local-store but need distinct stores + // so ArcFace 512-D and ECAPA-TDNN 192-D don't collide). Namespace + // isolation is already handled upstream: ModelLoader spawns a fresh + // local-store process per (backend, model) tuple, so each namespace is + // its own Store{} instance. Nothing to do here beyond accepting the load. + _ = opts return nil } diff --git a/backend/python/insightface/engines.py b/backend/python/insightface/engines.py index 5055d503e..b8c814cf6 100644 --- a/backend/python/insightface/engines.py +++ b/backend/python/insightface/engines.py @@ -173,6 +173,30 @@ def _build_antispoofer(options: dict[str, str], model_dir: str | None) -> Antisp # ─── InsightFaceEngine ──────────────────────────────────────────────── +# Canonical ONNX manifest for each upstream insightface pack (v0.7 release +# at github.com/deepinsight/insightface/releases). LocalAI's gallery extracts +# these zips flat into the models directory, so when multiple packs or other +# backends drop their own ONNX files alongside, the glob-the-directory +# approach picks up foreign files and insightface's model_zoo.get_model() +# raises IndexError trying to index `input_shape[2]` on a tensor that isn't +# shaped like a face model. The manifest lets us pre-filter to only the +# files that actually belong to the requested pack — deterministic, correct +# pack choice, no crashes on neighbour ONNX files. +_KNOWN_PACK_MANIFESTS: dict[str, frozenset[str]] = { + "buffalo_l": frozenset({ + "det_10g.onnx", + "w600k_r50.onnx", + "genderage.onnx", + "2d106det.onnx", + "1k3d68.onnx", + }), + "buffalo_sc": frozenset({ + "det_500m.onnx", + "w600k_mbf.onnx", + }), +} + + class InsightFaceEngine: """Drives insightface's model_zoo directly — no FaceAnalysis wrapper. @@ -222,6 +246,21 @@ class InsightFaceEngine: ) onnx_files = sorted(glob.glob(os.path.join(pack_dir, "*.onnx"))) + # When the pack extracts flat into a shared models directory it + # mixes with ONNX files from other backends (opencv face engine, + # MiniFASNet antispoof, WeSpeaker voice embedding, other buffalo + # packs installed earlier). Feeding those into model_zoo.get_model() + # blows up inside insightface's router — it assumes a 4-D NCHW + # input and indexes `input_shape[2]` on tensors that aren't shaped + # like a face model, raising IndexError. For the upstream packs we + # know the exact ONNX manifest; scoping to it makes the load + # deterministic (without it, det_10g.onnx from buffalo_l sorts + # before det_500m.onnx from buffalo_sc and silently wins). + manifest = _KNOWN_PACK_MANIFESTS.get(self.model_pack) + if manifest is not None: + scoped = [f for f in onnx_files if os.path.basename(f) in manifest] + if scoped: + onnx_files = scoped if not onnx_files: raise ValueError(f"no ONNX files in pack directory: {pack_dir}") @@ -231,14 +270,31 @@ class InsightFaceEngine: self._providers = ["CUDAExecutionProvider", "CPUExecutionProvider"] self.models = {} + skipped: list[tuple[str, str]] = [] for onnx_file in onnx_files: - m = model_zoo.get_model(onnx_file, providers=self._providers) + try: + m = model_zoo.get_model(onnx_file, providers=self._providers) + except Exception as err: + # Foreign ONNX (wrong rank/shape, non-insightface model) — + # older insightface versions raise IndexError / ValueError + # instead of returning None. Keep loading the rest. + skipped.append((os.path.basename(onnx_file), str(err))) + continue if m is None: + skipped.append((os.path.basename(onnx_file), "unknown taskname")) continue # First occurrence of each taskname wins (matches FaceAnalysis). if m.taskname not in self.models: self.models[m.taskname] = m + if skipped: + import sys + print( + f"[insightface] skipped {len(skipped)} non-pack ONNX file(s) in {pack_dir}: " + + ", ".join(f"{n} ({why})" for n, why in skipped), + file=sys.stderr, + ) + if "detection" not in self.models: raise ValueError(f"no detector (taskname='detection') found in {pack_dir}") self.det_model = self.models["detection"] diff --git a/backend/python/speaker-recognition/engines.py b/backend/python/speaker-recognition/engines.py index ef52f0247..85df80bec 100644 --- a/backend/python/speaker-recognition/engines.py +++ b/backend/python/speaker-recognition/engines.py @@ -317,8 +317,23 @@ class OnnxDirectEngine: else: provider_list = ["CPUExecutionProvider"] self._session = ort.InferenceSession(onnx_path, providers=provider_list) - self._input_name = self._session.get_inputs()[0].name + input_meta = self._session.get_inputs()[0] + self._input_name = input_meta.name + # Pre-exported speaker encoders come in two shapes: + # rank-2 [batch, samples] — some 3D-Speaker exports feed raw waveform. + # rank-3 [batch, frames, n_mels] — WeSpeaker and most Kaldi-lineage encoders + # expect pre-computed Kaldi FBank features. + # We detect this at load time and branch in embed(), because feeding raw audio + # into a rank-3 graph is exactly what triggered + # "Invalid rank for input: feats Got: 2 Expected: 3". + self._input_rank = len(input_meta.shape) if input_meta.shape is not None else 2 self._expected_sr = int(options.get("sample_rate", "16000")) + self._fbank_mels = int(options.get("fbank_num_mel_bins", "80")) + self._fbank_frame_length_ms = float(options.get("fbank_frame_length_ms", "25")) + self._fbank_frame_shift_ms = float(options.get("fbank_frame_shift_ms", "10")) + # Per-utterance cepstral mean normalisation — on for WeSpeaker by default, + # toggleable for encoders that expect raw FBank. + self._fbank_cmn = options.get("fbank_cmn", "true").lower() in ("1", "true", "yes") self._analysis = AnalysisHead(options) def _load_waveform(self, path: str): @@ -344,11 +359,37 @@ class OnnxDirectEngine: import numpy as np audio = self._load_waveform(audio_path) - feed = audio.reshape(1, -1) + if self._input_rank >= 3: + feats = self._extract_fbank(audio) # [frames, n_mels] + feed = feats[np.newaxis, :, :] # [1, frames, n_mels] + else: + feed = audio.reshape(1, -1) # [1, samples] out = self._session.run(None, {self._input_name: feed}) vec = np.asarray(out[0]).reshape(-1) return [float(x) for x in vec] + def _extract_fbank(self, audio): + """Compute Kaldi-style 80-dim FBank features for speaker encoders that + expect pre-featurised input (WeSpeaker, most 3D-Speaker exports). + torchaudio is already a backend dependency for SpeechBrain — no new + package required.""" + import numpy as np + import torch # type: ignore + import torchaudio.compliance.kaldi as kaldi # type: ignore + + tensor = torch.from_numpy(audio).unsqueeze(0) # [1, samples] + feats = kaldi.fbank( + tensor, + sample_frequency=self._expected_sr, + num_mel_bins=self._fbank_mels, + frame_length=self._fbank_frame_length_ms, + frame_shift=self._fbank_frame_shift_ms, + dither=0.0, + ) # [frames, n_mels] + if self._fbank_cmn: + feats = feats - feats.mean(dim=0, keepdim=True) + return feats.numpy().astype(np.float32) + def compare(self, audio1: str, audio2: str) -> float: return _cosine_distance(self.embed(audio1), self.embed(audio2)) diff --git a/core/application/application.go b/core/application/application.go index b1c4ef86a..22162fd19 100644 --- a/core/application/application.go +++ b/core/application/application.go @@ -81,18 +81,30 @@ func newApplication(appConfig *config.ApplicationConfig) *Application { // The resolver closes over the ModelLoader so the Registry stays // decoupled from loader plumbing; swapping in a postgres-backed // implementation later is a single construction change here. + // + // `faceStoreName` is the default namespace passed to StoreBackend when + // the request doesn't override it. Face and voice MUST use distinct + // namespaces — the local-store gRPC surface rejects mixed dimensions + // inside one namespace ("Try to add key with length N when existing + // length is M"). ArcFace buffalo_l produces 512-dim embeddings while + // ECAPA-TDNN produces 192-dim; enrolling one after the other into a + // shared namespace is exactly how we hit that error. + const ( + faceStoreName = "localai-face-biometrics" + voiceStoreName = "localai-voice-biometrics" + ) faceStoreResolver := func(_ context.Context, storeName string) (pkggrpc.Backend, error) { return corebackend.StoreBackend(ml, appConfig, storeName, "") } - app.faceRegistry = facerecognition.NewStoreRegistry(faceStoreResolver, "", faceEmbeddingDim) + app.faceRegistry = facerecognition.NewStoreRegistry(faceStoreResolver, faceStoreName, faceEmbeddingDim) // Voice (speaker) recognition registry — same plumbing, separate - // registry so embedding spaces stay isolated (a face vector and a - // speaker vector are not comparable). + // namespace so embedding spaces stay isolated (a face vector and a + // speaker vector are not comparable and differ in dimensionality). voiceStoreResolver := func(_ context.Context, storeName string) (pkggrpc.Backend, error) { return corebackend.StoreBackend(ml, appConfig, storeName, "") } - app.voiceRegistry = voicerecognition.NewStoreRegistry(voiceStoreResolver, "", voiceEmbeddingDim) + app.voiceRegistry = voicerecognition.NewStoreRegistry(voiceStoreResolver, voiceStoreName, voiceEmbeddingDim) return app } diff --git a/core/application/startup.go b/core/application/startup.go index 241ea8b22..b0484d0a9 100644 --- a/core/application/startup.go +++ b/core/application/startup.go @@ -242,6 +242,12 @@ func New(opts ...config.AppOption) (*Application, error) { bmFn := func() galleryop.BackendManager { return application.GalleryService().BackendManager() } uc := NewUpgradeChecker(options, application.ModelLoader(), application.distributedDB(), bmFn) application.upgradeChecker = uc + // Refresh the upgrade cache the moment a backend op finishes — otherwise + // the UI keeps showing a just-upgraded backend as upgradeable until the + // next 6-hour tick. TriggerCheck is non-blocking. + if gs := application.GalleryService(); gs != nil { + gs.OnBackendOpCompleted = uc.TriggerCheck + } go uc.Run(options.Context) } diff --git a/core/backend/stores.go b/core/backend/stores.go index 78257180e..2fd4cc148 100644 --- a/core/backend/stores.go +++ b/core/backend/stores.go @@ -11,8 +11,17 @@ func StoreBackend(sl *model.ModelLoader, appConfig *config.ApplicationConfig, st if backend == "" { backend = model.LocalStoreBackend } + // ModelLoader caches backend processes by `modelID`, not by the `model` + // passed via WithModel. Without a distinct modelID, every StoreBackend + // call collapses to the same `modelID=""` cache slot — face (512-D) and + // voice (192-D) biometrics would then share the same local-store process + // and the second enrollment would fail with + // Try to add key with length N when existing length is M + // Use the store namespace as modelID so each namespace gets its own + // process instance and its own in-memory Store{}. sc := []model.Option{ model.WithBackendString(backend), + model.WithModelID(storeName), model.WithModel(storeName), } diff --git a/core/gallery/backends.go b/core/gallery/backends.go index 6bf8c5d14..ca9b07dfd 100644 --- a/core/gallery/backends.go +++ b/core/gallery/backends.go @@ -194,6 +194,20 @@ func InstallBackend(ctx context.Context, systemState *system.SystemState, modelL name := config.Name backendPath := filepath.Join(systemState.Backend.BackendsPath, name) + // Clean up legacy flat-layout artefacts: earlier dev builds of the + // golang backends dropped the compiled binary directly at + // `/` (a plain file) instead of + // `//` (the nested layout the current code + // expects). MkdirAll below returns ENOTDIR when such a stale file + // exists, permanently blocking any reinstall or upgrade. Remove the + // file first so the install can proceed; the new install will write + // the correct nested layout, including metadata.json + run.sh. + if fi, statErr := os.Lstat(backendPath); statErr == nil && !fi.IsDir() { + xlog.Warn("removing stale non-directory backend artefact to make room for fresh install", "path", backendPath) + if rmErr := os.Remove(backendPath); rmErr != nil { + return fmt.Errorf("failed to remove stale backend artefact at %s: %w", backendPath, rmErr) + } + } err = os.MkdirAll(backendPath, 0750) if err != nil { return fmt.Errorf("failed to create base path: %v", err) diff --git a/core/http/endpoints/localai/audio.go b/core/http/endpoints/localai/audio.go index f9da79859..e8a43b04c 100644 --- a/core/http/endpoints/localai/audio.go +++ b/core/http/endpoints/localai/audio.go @@ -14,7 +14,13 @@ import ( "github.com/mudler/LocalAI/pkg/utils" ) -var audioDataURIPattern = regexp.MustCompile(`^data:([^;]+);base64,`) +// Match `data:[;param=value...];base64,` — MediaRecorder in the browser +// produces data URIs like `data:audio/webm;codecs=opus;base64,...`, so the +// pre-`;base64,` section can contain zero or more parameter segments. The +// old `([^;]+)` form only matched exactly one segment and left recordings +// from the React UI's live-capture tab unparsed, which then failed base64 +// decoding on the leading `data:` bytes. +var audioDataURIPattern = regexp.MustCompile(`^data:[^,]+?;base64,`) var audioDownloadClient = http.Client{Timeout: 30 * time.Second} diff --git a/core/http/react-ui/src/App.css b/core/http/react-ui/src/App.css index 03c448243..debb7bca7 100644 --- a/core/http/react-ui/src/App.css +++ b/core/http/react-ui/src/App.css @@ -4806,6 +4806,1266 @@ select.input { justify-content: center; } +/* ──────────────────── Biometrics (face + voice recognition) ──────────────────── */ + +.biometrics-page { + padding: var(--spacing-xl); + max-width: 1320px; + margin: 0 auto; + width: 100%; + animation: fadeIn var(--duration-normal) var(--ease-default); +} + +.biometrics-page__header { + display: grid; + grid-template-columns: 1fr minmax(240px, 320px); + gap: var(--spacing-lg); + align-items: end; + margin-bottom: var(--spacing-lg); + padding-bottom: var(--spacing-md); + border-bottom: 1px solid var(--color-border-divider); +} + +.biometrics-page__header .page-title i { + color: var(--color-accent); +} + +.biometrics-page__model { + display: flex; + flex-direction: column; + gap: var(--spacing-xs); +} + +.biometrics-page__body { + display: flex; + flex-direction: column; + gap: var(--spacing-lg); + min-width: 0; +} + +@media (max-width: 720px) { + .biometrics-page__header { + grid-template-columns: 1fr; + align-items: stretch; + } +} + +/* Tabs — flat, underlined, inherit page tone */ +.biometrics-tabs { + display: flex; + gap: var(--spacing-xs); + border-bottom: 1px solid var(--color-border-subtle); + overflow-x: auto; + scrollbar-width: none; +} +.biometrics-tabs::-webkit-scrollbar { display: none; } + +.biometrics-tab { + background: transparent; + border: 0; + padding: var(--spacing-sm) var(--spacing-md); + color: var(--color-text-secondary); + font: inherit; + font-weight: var(--font-weight-medium); + cursor: pointer; + display: inline-flex; + align-items: center; + gap: var(--spacing-xs); + border-bottom: 2px solid transparent; + min-height: 44px; + transition: color var(--duration-fast), border-color var(--duration-fast); + white-space: nowrap; +} +.biometrics-tab:hover { color: var(--color-text-primary); } +.biometrics-tab.active { + color: var(--color-text-primary); + border-bottom-color: var(--color-accent); +} +.biometrics-tab i { color: var(--color-accent); font-size: 0.9em; } + +/* Two-column workflow layout */ +.biometrics-twocol { + display: grid; + grid-template-columns: minmax(300px, 380px) 1fr; + gap: var(--spacing-lg); + align-items: start; + min-width: 0; +} +@media (max-width: 980px) { + .biometrics-twocol { grid-template-columns: 1fr; } +} + +.biometrics-panel { + background: var(--color-surface-raised); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-lg); + padding: var(--spacing-lg); + box-shadow: var(--shadow-subtle), var(--shadow-inset-top); + display: flex; + flex-direction: column; + gap: var(--spacing-md); +} + +.biometrics-panel__title { + font-size: var(--text-lg); + font-weight: var(--font-weight-semibold); + margin: 0; + color: var(--color-text-primary); + display: flex; + align-items: center; + gap: var(--spacing-sm); +} +.biometrics-panel__title i { color: var(--color-accent); } +.biometrics-panel__note { + margin: 0; + font-size: var(--text-sm); + color: var(--color-text-secondary); + line-height: var(--leading-normal); +} + +.biometrics-results { + min-width: 0; + display: flex; + flex-direction: column; + gap: var(--spacing-md); +} + +.biometrics-empty { + background: var(--color-bg-secondary); + border: 1px dashed var(--color-border-default); + border-radius: var(--radius-lg); + padding: var(--spacing-2xl) var(--spacing-lg); + text-align: center; + min-height: 300px; + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + gap: var(--spacing-sm); + color: var(--color-text-secondary); +} +.biometrics-empty > i { + font-size: 2.5rem; + color: var(--color-accent); + opacity: 0.6; +} +.biometrics-empty h3 { + margin: 0; + font-size: var(--text-lg); + font-weight: var(--font-weight-semibold); + color: var(--color-text-primary); +} +.biometrics-empty p { + margin: 0; + max-width: 48ch; + line-height: var(--leading-normal); + font-size: var(--text-sm); +} + +/* Media input — file / webcam / record switcher */ +.biometrics-mediainput { + display: flex; + flex-direction: column; + gap: var(--spacing-xs); +} +.biometrics-mediainput__tabs { + display: inline-flex; + gap: 2px; + padding: 2px; + background: var(--color-bg-tertiary); + border-radius: var(--radius-md); + align-self: flex-start; +} +.biometrics-mediainput__tab { + background: transparent; + border: 0; + font: inherit; + color: var(--color-text-secondary); + padding: 6px 12px; + min-height: 32px; + border-radius: var(--radius-sm); + cursor: pointer; + display: inline-flex; + align-items: center; + gap: 6px; + font-size: var(--text-xs); + font-weight: var(--font-weight-medium); + transition: background var(--duration-fast), color var(--duration-fast); +} +.biometrics-mediainput__tab:hover:not(:disabled) { color: var(--color-text-primary); } +.biometrics-mediainput__tab.active { + background: var(--color-surface-raised); + color: var(--color-text-primary); + box-shadow: var(--shadow-subtle); +} +.biometrics-mediainput__tab:disabled { opacity: 0.4; cursor: not-allowed; } + +.biometrics-mediainput__body { + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} + +.biometrics-mediainput__live { + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} +.biometrics-mediainput__video { + width: 100%; + aspect-ratio: 4 / 3; + border-radius: var(--radius-md); + background: var(--color-surface-sunken); + object-fit: cover; +} +.biometrics-mediainput__controls { + display: flex; + gap: var(--spacing-xs); +} +.biometrics-mediainput__controls .btn { flex: 1; min-height: 40px; } + +.biometrics-mediainput__meter { + display: flex; + align-items: center; + gap: var(--spacing-sm); + padding: var(--spacing-sm) var(--spacing-md); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); + background: var(--color-bg-secondary); + color: var(--color-text-secondary); + font-size: var(--text-sm); + font-variant-numeric: tabular-nums; +} +.biometrics-mediainput__meter i { color: var(--color-text-muted); } +.biometrics-mediainput__meter.recording { + border-color: var(--color-error-border); + color: var(--color-text-primary); +} +.biometrics-mediainput__meter.recording i { + color: var(--color-error); + animation: biometrics-pulse 1.2s ease-in-out infinite; +} +@keyframes biometrics-pulse { + 0%, 100% { opacity: 1; } + 50% { opacity: 0.35; } +} + +.biometrics-mediainput__error { + margin: 0; + color: var(--color-error); + font-size: var(--text-sm); +} + +.biometrics-mediainput__notice { + display: flex; + gap: var(--spacing-sm); + align-items: flex-start; + padding: var(--spacing-sm) var(--spacing-md); + background: var(--color-warning-light); + border: 1px solid var(--color-warning-border); + border-radius: var(--radius-md); + color: var(--color-text-primary); + font-size: var(--text-sm); + line-height: var(--leading-normal); +} +.biometrics-mediainput__notice > i { + color: var(--color-warning); + margin-top: 3px; + flex-shrink: 0; +} +.biometrics-mediainput__notice strong { + display: block; + margin-bottom: 2px; +} +.biometrics-mediainput__notice p { + margin: 0; + color: var(--color-text-secondary); + font-size: var(--text-xs); +} +.biometrics-mediainput__notice code { + background: var(--color-bg-tertiary); + padding: 1px 6px; + border-radius: var(--radius-sm); + font-size: 0.95em; +} + +.biometrics-mediainput__preview { + display: flex; + flex-direction: column; + gap: var(--spacing-xs); + padding: var(--spacing-sm); + background: var(--color-bg-secondary); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); +} +.biometrics-mediainput__preview img { + width: 100%; + max-height: 220px; + object-fit: contain; + border-radius: var(--radius-sm); + background: var(--color-surface-sunken); +} +.biometrics-mediainput__preview audio { width: 100%; } + +.biometrics-mediainput__preview-meta { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--spacing-xs); +} +.biometrics-mediainput__source-pill { + display: inline-flex; + align-items: center; + gap: 6px; + font-size: var(--text-xs); + color: var(--color-text-muted); + max-width: 100%; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.biometrics-mediainput__clear { + background: transparent; + border: 0; + color: var(--color-text-muted); + cursor: pointer; + min-width: 32px; + min-height: 32px; + border-radius: var(--radius-sm); + transition: color var(--duration-fast), background var(--duration-fast); +} +.biometrics-mediainput__clear:hover { + color: var(--color-error); + background: var(--color-error-light); +} + +/* Fieldsets + chip toggles (attribute actions) */ +.biometrics-fieldset { + border: 0; + padding: 0; + margin: 0; + display: flex; + flex-direction: column; + gap: var(--spacing-xs); +} +.biometrics-fieldset legend { + font-size: var(--text-xs); + font-weight: var(--font-weight-semibold); + color: var(--color-text-secondary); + text-transform: uppercase; + letter-spacing: 0.06em; + padding: 0; + margin: 0; +} +.biometrics-chipset { + display: flex; + flex-wrap: wrap; + gap: var(--spacing-xs); +} +.biometrics-chip { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 6px 12px; + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-full); + font-size: var(--text-xs); + color: var(--color-text-secondary); + cursor: pointer; + text-transform: capitalize; + transition: border-color var(--duration-fast), color var(--duration-fast), background var(--duration-fast); + min-height: 32px; +} +.biometrics-chip input { position: absolute; opacity: 0; pointer-events: none; } +.biometrics-chip:hover { color: var(--color-text-primary); } +.biometrics-chip.active { + border-color: var(--color-accent-border); + background: var(--color-accent-light); + color: var(--color-text-primary); +} + +/* Toggle switch */ +.biometrics-switch { + display: inline-block; + position: relative; + width: 40px; + height: 22px; + flex-shrink: 0; +} +.biometrics-switch input { + position: absolute; + opacity: 0; + pointer-events: none; +} +.biometrics-switch > span { + position: absolute; + inset: 0; + background: var(--color-toggle-off); + border-radius: var(--radius-full); + transition: background var(--duration-fast); + cursor: pointer; +} +.biometrics-switch > span::after { + content: ""; + position: absolute; + left: 2px; + top: 2px; + width: 18px; + height: 18px; + border-radius: 50%; + background: #fff; + transition: transform var(--duration-fast); + box-shadow: var(--shadow-subtle); +} +.biometrics-switch input:checked + span { background: var(--color-accent); } +.biometrics-switch input:checked + span::after { transform: translateX(18px); } +.biometrics-switch input:focus-visible + span { + outline: 2px solid var(--color-border-focus); + outline-offset: 2px; +} + +/* Split view for analyze (image + summary side) */ +.biometrics-split { + display: grid; + grid-template-columns: minmax(0, 1.1fr) minmax(280px, 1fr); + gap: var(--spacing-md); + align-items: start; +} +@media (max-width: 980px) { + .biometrics-split { grid-template-columns: 1fr; } +} +.biometrics-split__media { + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} +.biometrics-split__aside { + display: flex; + flex-direction: column; + gap: var(--spacing-md); + min-width: 0; +} + +/* Bounding box overlay */ +.biometrics-bbox { + position: relative; + display: inline-block; + width: 100%; + max-width: 100%; + border-radius: var(--radius-md); + background: var(--color-surface-sunken); + overflow: hidden; + line-height: 0; +} +.biometrics-bbox img { + width: 100%; + height: auto; + display: block; +} +.biometrics-bbox__box { + position: absolute; + border: 2px solid var(--color-accent); + border-radius: 2px; + box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.25), 0 0 12px rgba(232, 168, 124, 0.35); + pointer-events: none; + transition: border-color var(--duration-fast); +} +.biometrics-bbox__box.tone-default { border-color: var(--color-border-strong); box-shadow: none; } +.biometrics-bbox__box.tone-success { border-color: var(--color-success); } +.biometrics-bbox__box.tone-error { border-color: var(--color-error); } +.biometrics-bbox__box.tone-warning { border-color: var(--color-warning); } +.biometrics-bbox__tag { + position: absolute; + left: -2px; + top: -2px; + transform: translateY(-100%); + background: var(--color-bg-overlay); + border: 1px solid var(--color-border-subtle); + border-bottom: 0; + border-radius: var(--radius-sm) var(--radius-sm) 0 0; + padding: 2px 8px; + font-size: var(--text-xs); + color: var(--color-text-primary); + display: inline-flex; + gap: 6px; + white-space: nowrap; + line-height: var(--leading-snug); +} +.biometrics-bbox__tag strong { font-weight: var(--font-weight-semibold); } +.biometrics-bbox__tag span { color: var(--color-text-secondary); } + +.biometrics-facepicker { + display: flex; + flex-wrap: wrap; + gap: var(--spacing-xs); +} +.biometrics-facepicker__chip { + background: var(--color-bg-secondary); + border: 1px solid var(--color-border-subtle); + color: var(--color-text-secondary); + padding: 4px 12px; + border-radius: var(--radius-full); + cursor: pointer; + font-size: var(--text-xs); + font: inherit; + font-size: var(--text-xs); + min-height: 32px; + transition: border-color var(--duration-fast), color var(--duration-fast), background var(--duration-fast); +} +.biometrics-facepicker__chip:hover { color: var(--color-text-primary); } +.biometrics-facepicker__chip.active { + border-color: var(--color-accent-border); + background: var(--color-accent-light); + color: var(--color-text-primary); +} +.biometrics-facepicker__chip small { margin-left: 4px; color: var(--color-text-muted); } + +/* Summary card (dominant attributes) */ +.biometrics-summary { + padding: var(--spacing-md); +} +.biometrics-summary__head { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--spacing-sm); + margin-bottom: var(--spacing-sm); +} +.biometrics-summary__head h3 { + font-size: var(--text-base); + margin: 0; + font-weight: var(--font-weight-semibold); + display: flex; + align-items: center; + gap: var(--spacing-xs); +} +.biometrics-summary__head h3 i { color: var(--color-accent); } +.biometrics-summary__head h3 small { + color: var(--color-text-muted); + font-weight: var(--font-weight-regular); + font-size: var(--text-sm); + font-variant-numeric: tabular-nums; +} +.biometrics-summary__grid { + display: grid; + grid-template-columns: max-content 1fr; + column-gap: var(--spacing-md); + row-gap: 6px; + margin: 0; +} +.biometrics-summary__grid dt { + color: var(--color-text-muted); + font-size: var(--text-xs); + text-transform: uppercase; + letter-spacing: 0.06em; + align-self: center; +} +.biometrics-summary__grid dd { + margin: 0; + color: var(--color-text-primary); + font-weight: var(--font-weight-medium); +} + +/* Distribution bars */ +.biometrics-dist { + padding: var(--spacing-md); + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} +.biometrics-dist__head { + display: flex; + align-items: center; + gap: var(--spacing-xs); +} +.biometrics-dist__head h3 { + font-size: var(--text-sm); + margin: 0; + font-weight: var(--font-weight-semibold); + letter-spacing: -0.005em; +} +.biometrics-dist__head i { color: var(--color-accent); } +.biometrics-dist__dominant { + margin-left: auto; + font-size: var(--text-xs); + color: var(--color-text-muted); + text-transform: capitalize; +} +.biometrics-dist__rows { + list-style: none; + padding: 0; + margin: 0; + display: flex; + flex-direction: column; + gap: 4px; +} +.biometrics-dist__row { + display: grid; + grid-template-columns: minmax(80px, 110px) 1fr max-content; + align-items: center; + gap: var(--spacing-sm); + font-size: var(--text-xs); +} +.biometrics-dist__label { + color: var(--color-text-secondary); + text-transform: capitalize; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.biometrics-dist__bar-wrap { + height: 6px; + background: var(--color-bg-tertiary); + border-radius: var(--radius-full); + overflow: hidden; +} +.biometrics-dist__bar { + height: 100%; + background: var(--color-text-muted); + border-radius: var(--radius-full); + transition: width var(--duration-normal) var(--ease-default); +} +.biometrics-dist__row.dominant .biometrics-dist__label { color: var(--color-text-primary); } +.biometrics-dist__row.dominant .biometrics-dist__bar { background: var(--color-accent); } +.biometrics-dist__value { + font-variant-numeric: tabular-nums; + color: var(--color-text-muted); + font-size: var(--text-xs); +} +.biometrics-dist__row.dominant .biometrics-dist__value { color: var(--color-text-primary); } + +/* Pill chips (liveness) */ +.biometrics-pill { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 4px 10px; + border-radius: var(--radius-full); + font-size: var(--text-xs); + font-weight: var(--font-weight-medium); + border: 1px solid var(--color-border-subtle); + background: var(--color-bg-secondary); + color: var(--color-text-secondary); +} +.biometrics-pill small { + color: var(--color-text-muted); + font-variant-numeric: tabular-nums; +} +.biometrics-pill.good { + background: var(--color-success-light); + border-color: var(--color-success-border); + color: var(--color-success); +} +.biometrics-pill.bad { + background: var(--color-error-light); + border-color: var(--color-error-border); + color: var(--color-error); +} +.biometrics-pill.muted { color: var(--color-text-muted); } + +/* Compare view */ +.biometrics-compare { + display: grid; + grid-template-columns: 1fr minmax(280px, 360px) 1fr; + gap: var(--spacing-md); + align-items: stretch; +} +@media (max-width: 1080px) { + .biometrics-compare { grid-template-columns: 1fr; } +} +.biometrics-compare__panel { + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} +.biometrics-compare__label { + font-size: var(--text-xs); + font-weight: var(--font-weight-semibold); + text-transform: uppercase; + letter-spacing: 0.06em; + color: var(--color-text-muted); +} +.biometrics-compare__center { + display: flex; + flex-direction: column; + gap: var(--spacing-md); + justify-content: center; +} +.biometrics-compare__threshold { + display: flex; + flex-direction: column; + gap: var(--spacing-xs); + background: var(--color-surface-raised); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); + padding: var(--spacing-sm) var(--spacing-md); +} +.biometrics-compare__threshold label { + display: flex; + justify-content: space-between; + align-items: center; + font-size: var(--text-sm); + font-weight: var(--font-weight-medium); +} +.biometrics-compare__threshold code { + color: var(--color-accent); + font-variant-numeric: tabular-nums; +} +.biometrics-compare__threshold input[type="range"] { + width: 100%; + accent-color: var(--color-accent); +} +.biometrics-compare__hint { + margin: 0; + color: var(--color-text-muted); + font-size: var(--text-xs); +} +.biometrics-compare__hint code { color: var(--color-text-secondary); } + +/* Match gauge */ +.biometrics-gauge { + background: var(--color-surface-raised); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-lg); + padding: var(--spacing-md); + display: flex; + flex-direction: column; + gap: var(--spacing-sm); + box-shadow: var(--shadow-subtle), var(--shadow-inset-top); +} +.biometrics-gauge__head { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--spacing-sm); +} +.biometrics-gauge__verdict { + display: inline-flex; + align-items: center; + gap: 8px; + font-size: var(--text-lg); + font-weight: var(--font-weight-semibold); +} +.biometrics-gauge.tone-success .biometrics-gauge__verdict { color: var(--color-success); } +.biometrics-gauge.tone-error .biometrics-gauge__verdict { color: var(--color-error); } +.biometrics-gauge__confidence { + text-align: right; + font-variant-numeric: tabular-nums; + line-height: var(--leading-tight); +} +.biometrics-gauge__confidence strong { + display: block; + font-size: var(--text-xl); + color: var(--color-text-primary); +} +.biometrics-gauge__confidence span { + font-size: var(--text-xs); + color: var(--color-text-muted); + text-transform: uppercase; + letter-spacing: 0.06em; +} +.biometrics-gauge__track { + position: relative; + height: 18px; + background: var(--color-bg-tertiary); + border-radius: var(--radius-full); + overflow: hidden; +} +.biometrics-gauge__zone { + position: absolute; + top: 0; + bottom: 0; + transition: width var(--duration-normal) var(--ease-default); +} +.biometrics-gauge__zone--match { + left: 0; + background: var(--color-success-light); + border-right: 1px dashed var(--color-success-border); +} +.biometrics-gauge__zone--miss { + background: var(--color-error-light); +} +.biometrics-gauge__threshold { + position: absolute; + top: 0; + bottom: 0; + width: 2px; + background: var(--color-border-strong); + transform: translateX(-1px); +} +.biometrics-gauge__threshold span { + position: absolute; + bottom: 100%; + left: 50%; + transform: translateX(-50%); + font-size: 9px; + text-transform: uppercase; + color: var(--color-text-muted); + letter-spacing: 0.08em; + padding: 1px 4px; + white-space: nowrap; +} +.biometrics-gauge__marker { + position: absolute; + top: -4px; + bottom: -4px; + width: 12px; + transform: translateX(-6px); + background: var(--color-text-primary); + border-radius: 2px; + border: 2px solid var(--color-surface-raised); + transition: left var(--duration-normal) var(--ease-default); + box-shadow: var(--shadow-sm); +} +.biometrics-gauge__marker span { + position: absolute; + top: 100%; + left: 50%; + transform: translateX(-50%); + font-size: 9px; + text-transform: uppercase; + color: var(--color-text-primary); + letter-spacing: 0.08em; + padding-top: 4px; + white-space: nowrap; +} +.biometrics-gauge__footer { + display: flex; + justify-content: space-between; + gap: var(--spacing-md); + font-size: var(--text-xs); + color: var(--color-text-muted); +} +.biometrics-gauge__footer em { + text-transform: uppercase; + letter-spacing: 0.06em; + font-style: normal; + margin-right: 4px; +} +.biometrics-gauge__footer code { + font-variant-numeric: tabular-nums; + color: var(--color-text-secondary); +} + +/* Waveform */ +.biometrics-waveform { + --biometrics-wave: var(--color-accent); + position: relative; + width: 100%; + background: var(--color-surface-sunken); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); + overflow: hidden; +} +.biometrics-waveform--error { + padding: var(--spacing-md); + color: var(--color-error); + font-size: var(--text-sm); +} +.biometrics-waveform__segment { + position: absolute; + top: 0; + bottom: 0; + background: rgba(232, 168, 124, 0.16); + border-left: 1px dashed var(--color-accent-border); + border-right: 1px dashed var(--color-accent-border); + pointer-events: none; +} +.biometrics-waveform__segment.tone-info { background: var(--color-info-light); border-color: var(--color-info-border); } +.biometrics-waveform__segment.tone-success { background: var(--color-success-light); border-color: var(--color-success-border); } +.biometrics-waveform__segment.tone-warning { background: var(--color-warning-light); border-color: var(--color-warning-border); } +.biometrics-waveform__segment.tone-accent { background: var(--color-accent-light); border-color: var(--color-accent-border); } +.biometrics-waveform__seglabel { + position: absolute; + top: 4px; + left: 4px; + font-size: var(--text-xs); + color: var(--color-text-primary); + background: var(--color-bg-overlay); + padding: 1px 6px; + border-radius: var(--radius-sm); + max-width: calc(100% - 8px); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.biometrics-waveform__duration { + position: absolute; + right: 8px; + bottom: 6px; + font-size: 11px; + color: var(--color-text-muted); + font-variant-numeric: tabular-nums; + background: var(--color-bg-overlay); + padding: 1px 6px; + border-radius: var(--radius-sm); +} +.biometrics-waveform__loading { + position: absolute; + inset: 0; + display: flex; + align-items: center; + justify-content: center; + color: var(--color-text-muted); + font-size: var(--text-sm); +} + +/* Enrollment layout (register + identify + list) */ +.biometrics-enrollgrid { + display: grid; + grid-template-columns: minmax(300px, 1fr) minmax(300px, 1fr); + grid-template-areas: + "register identify" + "list list"; + gap: var(--spacing-lg); +} +.biometrics-enrollgrid__register { grid-area: register; } +.biometrics-enrollgrid__identify { grid-area: identify; } +.biometrics-enrollgrid__list { grid-area: list; min-width: 0; } +@media (max-width: 980px) { + .biometrics-enrollgrid { + grid-template-columns: 1fr; + grid-template-areas: + "register" + "identify" + "list"; + } +} +.biometrics-enrollgrid__register form, +.biometrics-enrollgrid__identify form { + display: flex; + flex-direction: column; + gap: var(--spacing-md); +} +.biometrics-enrollgrid__err { + margin-top: var(--spacing-sm); +} + +.biometrics-enroll__head { + display: flex; + align-items: center; + justify-content: space-between; + margin-bottom: var(--spacing-md); +} +.biometrics-enroll__count { + background: var(--color-bg-tertiary); + color: var(--color-text-secondary); + font-size: var(--text-xs); + font-weight: var(--font-weight-medium); + padding: 2px 8px; + border-radius: var(--radius-full); + margin-left: var(--spacing-xs); +} + +.biometrics-enroll__grid { + list-style: none; + padding: 0; + margin: 0; + display: grid; + grid-template-columns: repeat(auto-fill, minmax(220px, 1fr)); + gap: var(--spacing-md); +} +.biometrics-enroll__card { + position: relative; + background: var(--color-surface-raised); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-lg); + padding: var(--spacing-md); + display: flex; + flex-direction: column; + gap: var(--spacing-sm); + transition: border-color var(--duration-fast), transform var(--duration-fast); +} +.biometrics-enroll__card:hover { + border-color: var(--color-border-default); + transform: translateY(-1px); +} +.biometrics-enroll__card.highlight { + border-color: var(--color-accent-border); + box-shadow: 0 0 0 1px var(--color-accent-border); + animation: biometrics-highlight 1.4s ease-out; +} +@keyframes biometrics-highlight { + 0% { box-shadow: 0 0 0 4px var(--color-accent-light); } + 100% { box-shadow: 0 0 0 1px var(--color-accent-border); } +} + +.biometrics-enroll__media { + aspect-ratio: 1 / 1; + background: var(--color-surface-sunken); + border-radius: var(--radius-md); + overflow: hidden; + display: flex; + align-items: center; + justify-content: center; +} +.biometrics-enroll__media img { + width: 100%; + height: 100%; + object-fit: cover; +} +.biometrics-enroll__media audio { + width: 90%; +} +.biometrics-enroll__initials { + font-size: 2rem; + font-weight: var(--font-weight-semibold); + color: var(--color-text-muted); + letter-spacing: 0.04em; +} +.biometrics-enroll__body { display: flex; flex-direction: column; gap: 4px; } +.biometrics-enroll__name { + font-weight: var(--font-weight-semibold); + font-size: var(--text-sm); + color: var(--color-text-primary); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.biometrics-enroll__labels { + list-style: none; + padding: 0; + margin: 0; + display: flex; + flex-wrap: wrap; + gap: 4px; +} +.biometrics-enroll__labels li { + font-size: var(--text-xs); + color: var(--color-text-secondary); + background: var(--color-bg-secondary); + padding: 2px 6px; + border-radius: var(--radius-sm); +} +.biometrics-enroll__labels li span { + color: var(--color-text-muted); + margin-right: 4px; +} +.biometrics-enroll__meta { + font-size: var(--text-xs); + color: var(--color-text-muted); + display: inline-flex; + align-items: center; + gap: 4px; +} +.biometrics-enroll__delete { + position: absolute; + top: 8px; + right: 8px; + background: var(--color-bg-overlay); + border: 1px solid var(--color-border-subtle); + color: var(--color-text-muted); + border-radius: var(--radius-sm); + width: 28px; + height: 28px; + cursor: pointer; + display: inline-flex; + align-items: center; + justify-content: center; + opacity: 0; + transition: opacity var(--duration-fast), color var(--duration-fast), background var(--duration-fast); +} +.biometrics-enroll__card:hover .biometrics-enroll__delete, +.biometrics-enroll__card:focus-within .biometrics-enroll__delete { opacity: 1; } +.biometrics-enroll__delete:hover { + color: var(--color-error); + background: var(--color-error-light); + border-color: var(--color-error-border); +} +.biometrics-enroll__empty { + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + gap: var(--spacing-sm); + padding: var(--spacing-xl); + border: 1px dashed var(--color-border-default); + border-radius: var(--radius-lg); + text-align: center; + color: var(--color-text-secondary); + background: var(--color-bg-secondary); +} +.biometrics-enroll__empty > i { + font-size: 2rem; + color: var(--color-accent); + opacity: 0.6; +} +.biometrics-enroll__empty p { + margin: 0; + max-width: 44ch; + line-height: var(--leading-normal); + font-size: var(--text-sm); +} + +/* Matches list (identify results) */ +.biometrics-matches { + list-style: none; + padding: 0; + margin: 0; + display: flex; + flex-direction: column; + gap: var(--spacing-sm); +} +.biometrics-matches__empty { + padding: var(--spacing-md); + border: 1px dashed var(--color-border-default); + border-radius: var(--radius-md); + color: var(--color-text-muted); + text-align: center; + font-size: var(--text-sm); +} +.biometrics-matches__row { + display: grid; + grid-template-columns: 32px 56px 1fr; + gap: var(--spacing-sm); + align-items: center; + padding: var(--spacing-sm); + background: var(--color-bg-secondary); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); +} +.biometrics-matches__row.match { border-color: var(--color-success-border); } +.biometrics-matches__rank { + font-size: var(--text-xs); + color: var(--color-text-muted); + font-weight: var(--font-weight-semibold); + text-align: center; +} +.biometrics-matches__avatar { + width: 56px; + height: 56px; + border-radius: var(--radius-md); + overflow: hidden; + background: var(--color-surface-sunken); + display: flex; + align-items: center; + justify-content: center; + color: var(--color-text-muted); + font-weight: var(--font-weight-semibold); + font-size: var(--text-sm); +} +.biometrics-matches__avatar img { width: 100%; height: 100%; object-fit: cover; } +.biometrics-matches__body { min-width: 0; display: flex; flex-direction: column; gap: 4px; } +.biometrics-matches__name { + display: flex; + align-items: center; + gap: var(--spacing-xs); + font-size: var(--text-sm); + min-width: 0; +} +.biometrics-matches__name strong { + font-weight: var(--font-weight-semibold); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.biometrics-matches__badge { + font-size: 10px; + text-transform: uppercase; + letter-spacing: 0.06em; + padding: 2px 6px; + border-radius: var(--radius-sm); + background: var(--color-bg-tertiary); + color: var(--color-text-muted); + display: inline-flex; + align-items: center; + gap: 4px; +} +.biometrics-matches__badge.match { + background: var(--color-success-light); + color: var(--color-success); +} +.biometrics-matches__meter { + height: 4px; + background: var(--color-bg-tertiary); + border-radius: var(--radius-full); + overflow: hidden; +} +.biometrics-matches__fill { + height: 100%; + background: var(--color-accent); + transition: width var(--duration-normal) var(--ease-default); +} +.biometrics-matches__row.match .biometrics-matches__fill { background: var(--color-success); } +.biometrics-matches__meta { + display: flex; + gap: var(--spacing-md); + font-size: var(--text-xs); + color: var(--color-text-muted); +} +.biometrics-matches__meta code { + color: var(--color-text-secondary); + font-variant-numeric: tabular-nums; +} +.biometrics-matches__preview { width: 100%; } + +/* Embedding inspector */ +.biometrics-embed { + display: flex; + flex-direction: column; + gap: var(--spacing-sm); + padding: var(--spacing-md); +} +.biometrics-embed__head { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: var(--spacing-sm); +} +.biometrics-embed__title { + font-size: var(--text-base); + font-weight: var(--font-weight-semibold); +} +.biometrics-embed__meta { + display: flex; + flex-wrap: wrap; + gap: var(--spacing-md); + font-size: var(--text-xs); + color: var(--color-text-muted); + margin-top: 4px; +} +.biometrics-embed__meta strong { color: var(--color-text-primary); font-variant-numeric: tabular-nums; font-weight: var(--font-weight-semibold); } +.biometrics-embed__meta code { color: var(--color-text-secondary); } + +/* Response details pane */ +.biometrics-response { + background: var(--color-bg-secondary); + border: 1px solid var(--color-border-subtle); + border-radius: var(--radius-md); + overflow: hidden; +} +.biometrics-response summary { + padding: var(--spacing-sm) var(--spacing-md); + cursor: pointer; + font-size: var(--text-sm); + color: var(--color-text-secondary); + display: flex; + align-items: center; + gap: var(--spacing-xs); + list-style: none; + user-select: none; + min-height: 40px; +} +.biometrics-response summary::-webkit-details-marker { display: none; } +.biometrics-response summary i { transition: transform var(--duration-fast); } +.biometrics-response[open] summary i { transform: rotate(90deg); } +.biometrics-response pre { + margin: 0; + padding: var(--spacing-md); + background: var(--color-surface-sunken); + font-size: var(--text-xs); + color: var(--color-text-secondary); + overflow-x: auto; + max-height: 360px; + line-height: var(--leading-snug); +} + +.form-label__hint { + color: var(--color-text-muted); + font-weight: var(--font-weight-regular); + margin-left: 4px; +} + /* Reduced motion accessibility */ @media (prefers-reduced-motion: reduce) { *, *::before, *::after { diff --git a/core/http/react-ui/src/components/Sidebar.jsx b/core/http/react-ui/src/components/Sidebar.jsx index afd289e18..340405f81 100644 --- a/core/http/react-ui/src/components/Sidebar.jsx +++ b/core/http/react-ui/src/components/Sidebar.jsx @@ -24,6 +24,18 @@ const sections = [ { path: '/app/quantize', icon: 'fas fa-compress', label: 'Quantize (Experimental)', feature: 'quantization' }, ], }, + { + id: 'biometrics', + title: 'Biometrics', + featureMap: { + '/app/face': 'face_recognition', + '/app/voice': 'voice_recognition', + }, + items: [ + { path: '/app/face', icon: 'fas fa-face-smile', label: 'Face Recognition', feature: 'face_recognition' }, + { path: '/app/voice', icon: 'fas fa-microphone-lines', label: 'Voice Recognition', feature: 'voice_recognition' }, + ], + }, { id: 'agents', title: 'Agents', diff --git a/core/http/react-ui/src/components/biometrics/BoundingBoxCanvas.jsx b/core/http/react-ui/src/components/biometrics/BoundingBoxCanvas.jsx new file mode 100644 index 000000000..df72f7e8e --- /dev/null +++ b/core/http/react-ui/src/components/biometrics/BoundingBoxCanvas.jsx @@ -0,0 +1,63 @@ +import { useEffect, useRef, useState } from 'react' + +// BoundingBoxCanvas — overlay face-detection rectangles on the user-supplied image. +// boxes: [{ x, y, w, h, label?, sublabel?, tone? }] +// tone: 'default' | 'success' | 'warning' | 'error' | 'accent' +export default function BoundingBoxCanvas({ src, boxes = [], alt = '' }) { + const wrapRef = useRef(null) + const imgRef = useRef(null) + const [dims, setDims] = useState({ w: 0, h: 0, natW: 0, natH: 0 }) + + useEffect(() => { + const update = () => { + if (!wrapRef.current || !imgRef.current) return + const rect = imgRef.current.getBoundingClientRect() + setDims({ + w: rect.width, + h: rect.height, + natW: imgRef.current.naturalWidth || 1, + natH: imgRef.current.naturalHeight || 1, + }) + } + update() + const ro = new ResizeObserver(update) + if (imgRef.current) ro.observe(imgRef.current) + window.addEventListener('resize', update) + return () => { + ro.disconnect() + window.removeEventListener('resize', update) + } + }, [src]) + + const sx = dims.natW ? dims.w / dims.natW : 1 + const sy = dims.natH ? dims.h / dims.natH : 1 + + return ( +
+ {src && {alt} { + setDims({ + w: e.target.getBoundingClientRect().width, + h: e.target.getBoundingClientRect().height, + natW: e.target.naturalWidth, + natH: e.target.naturalHeight, + }) + }} />} + {boxes.map((b, i) => ( +
+ {(b.label || b.sublabel) && ( +
+ {b.label && {b.label}} + {b.sublabel && {b.sublabel}} +
+ )} +
+ ))} +
+ ) +} diff --git a/core/http/react-ui/src/components/biometrics/DistributionBars.jsx b/core/http/react-ui/src/components/biometrics/DistributionBars.jsx new file mode 100644 index 000000000..53c95f719 --- /dev/null +++ b/core/http/react-ui/src/components/biometrics/DistributionBars.jsx @@ -0,0 +1,33 @@ +// DistributionBars — one horizontal bar per label, width proportional to value. +// distribution: Record (values are probabilities 0..1 or any positive scale). +// dominant: string — highlighted row. +export default function DistributionBars({ title, distribution, dominant, icon }) { + if (!distribution || Object.keys(distribution).length === 0) return null + const entries = Object.entries(distribution).sort((a, b) => b[1] - a[1]) + const max = entries.reduce((m, [, v]) => Math.max(m, v), 0) || 1 + + return ( +
+
+ {icon &&