Files
LocalAI/core/http/react-ui/src/router.jsx
Richard Palethorpe 9058a2bb46 feat: Add 3d generation UI/API and trellis2cpp backend (#10979)
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint

Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:

- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
  glb dst, seed/step/cfg_scale/texture_steps, quality and background
  enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
  the backend interfaces; connection-evicting and distributed-node
  wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
  backend), '3d' canonical usecase string mapped to the Generate3D
  method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
  OpenAIResponse with a /generated-3d URL or b64_json; conditioning
  image accepted as URL, base64, or data URI; quality/background
  validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend

Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:

- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
  asserted at startup), eager pipeline load with model-set validation
  (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
  exactly like the upstream demo), Generate3D via t2_generate +
  t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
  cover resolution/validation/param mapping — CI never downloads the
  multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
  libggml sonames collide across variants, unlike sd-ggml's flat
  renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
  amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
  latest/master image entries, bump_deps tracking of the pbr-textures
  branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
  llama-cpp so the .gguf match isn't stolen) and expands any trellis
  URI to the full 10-file component set spanning the three LocalAI-io
  HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
  trellis2-4b-geometry (512 untextured) with verified sha256s.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(ui): 3D generation page with native GLB viewer and IndexedDB history

Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:

- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
  renderer (quaternion trackball, metallic-roughness PBR, ACES,
  hidden-line wireframe with a bounded index budget) and pairs it with
  a minimal GLB parser for the two forms t2_bake_glb emits — dense
  vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
  normalized integers) and the opt-in UV-atlas textured form. Parsing
  happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
  the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
  GLBs are multi-MB binaries localStorage can't hold — and the page
  offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
  /api/models/capabilities serves), threeDApi, router entries, Studio
  tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
  one-triangle vertex-PBR GLB through the parser/viewer and exercises
  IndexedDB persistence, selection, deletion, and API errors.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(3d): address API correctness and UX issues

Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.

Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(3d): add previewable print remeshing

Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.

Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* build(trellis2cpp): centralize remesh dependency pins

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(kokoros): implement Generate3D stub for new proto RPC

The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.

Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
2026-07-29 16:15:04 +02:00

216 lines
11 KiB
JavaScript

import { lazy } from 'react'
import { createBrowserRouter, Navigate, useParams } from 'react-router-dom'
import { routerBasename } from './utils/basePath'
import App from './App'
import RequireAdmin from './components/RequireAdmin'
import RequireAuth from './components/RequireAuth'
import RequireAuthEnabled from './components/RequireAuthEnabled'
import RequireFeature from './components/RequireFeature'
// Pages are code-split: each becomes its own chunk loaded on demand, so a route
// no longer drags every other page (and its heavy deps — CodeMirror, the MCP
// SDK, yaml, marked) into the initial bundle. The <Suspense> boundary in
// App.jsx (around <Outlet/>) shows nothing while a chunk loads, keeping the
// sidebar/header mounted.
//
// `page(key, loader)` registers the dynamic import under a route-segment key
// (the first segment after /app/) so a NavLink can warm the chunk on hover via
// `preloadRoute('/app/chat')`. Dynamic import() is memoised by the module
// loader, so a preloaded chunk is reused — not re-fetched — when the user
// actually navigates. Pages with `key: null` aren't sidebar-reachable; they
// still code-split, they just won't be preloaded from the nav.
const preloaders = {}
function page(key, loader) {
if (key !== null) preloaders[key] = loader
return lazy(loader)
}
export function preloadRoute(path) {
if (!path) return
const m = path.match(/^\/app(?:\/([^/?#]*))?/)
if (!m) return
preloaders[m[1] ?? '']?.().catch(() => { /* network blip — real click will retry */ })
}
const Home = page('', () => import('./pages/Home'))
const Chat = page('chat', () => import('./pages/Chat'))
const Models = page('models', () => import('./pages/Models'))
const Manage = page('manage', () => import('./pages/Manage'))
const ImageGen = page('image', () => import('./pages/ImageGen'))
const VideoGen = page('video', () => import('./pages/VideoGen'))
const ThreeDGen = page('3d', () => import('./pages/ThreeDGen'))
const TTS = page('tts', () => import('./pages/TTS'))
const Sound = page('sound', () => import('./pages/Sound'))
const AudioTransform = page('transform', () => import('./pages/AudioTransform'))
const Talk = page('talk', () => import('./pages/Talk'))
const Backends = page('backends', () => import('./pages/Backends'))
// Only referenced from JSX below, which eslint cannot see without
// eslint-plugin-react. Suppressed here rather than left to widen the file's
// warning count; the surrounding page consts predate the lint baseline.
// eslint-disable-next-line no-unused-vars
const Activity = page('activity', () => import('./pages/Activity'))
const Settings = page('settings', () => import('./pages/Settings'))
const Traces = page('traces', () => import('./pages/Traces'))
const P2P = page('p2p', () => import('./pages/P2P'))
const Agents = page('agents', () => import('./pages/Agents'))
const AgentCreate = page(null, () => import('./pages/AgentCreate'))
const AgentChat = page(null, () => import('./pages/AgentChat'))
const AgentStatus = page(null, () => import('./pages/AgentStatus'))
const Collections = page('collections', () => import('./pages/Collections'))
const CollectionDetails = page(null, () => import('./pages/CollectionDetails'))
const Skills = page('skills', () => import('./pages/Skills'))
const SkillEdit = page(null, () => import('./pages/SkillEdit'))
const AgentJobs = page('agent-jobs', () => import('./pages/AgentJobs'))
const AgentTaskDetails = page(null, () => import('./pages/AgentTaskDetails'))
const AgentJobDetails = page(null, () => import('./pages/AgentJobDetails'))
const ModelEditor = page(null, () => import('./pages/ModelEditor'))
// PipelineEditor removed — the Model Editor with templates handles all model types
const ImportModel = page(null, () => import('./pages/ImportModel'))
const BackendLogs = page(null, () => import('./pages/BackendLogs'))
const Explorer = page(null, () => import('./pages/Explorer'))
const Login = page(null, () => import('./pages/Login'))
const FineTune = page('fine-tune', () => import('./pages/FineTune'))
const Quantize = page('quantize', () => import('./pages/Quantize'))
const Studio = page('studio', () => import('./pages/Studio'))
const FaceRecognition = page('face', () => import('./pages/FaceRecognition'))
const VoiceRecognition = page('voice', () => import('./pages/VoiceRecognition'))
const VoiceLibrary = page('voice-library', () => import('./pages/VoiceLibrary'))
const VoiceProfileCreate = page(null, () => import('./pages/VoiceProfileCreate'))
const Nodes = page('nodes', () => import('./pages/Nodes'))
const Scheduling = page('scheduling', () => import('./pages/Scheduling'))
const NodeBackendLogs = page(null, () => import('./pages/NodeBackendLogs'))
const NodeDetail = page(null, () => import('./pages/NodeDetail'))
const NotFound = page(null, () => import('./pages/NotFound'))
const Usage = page('usage', () => import('./pages/Usage'))
const Users = page('users', () => import('./pages/Users'))
const Middleware = page('middleware', () => import('./pages/Middleware'))
const Account = page('account', () => import('./pages/Account'))
import ConsoleLayout from './components/console/ConsoleLayout'
import { buildConsole, operateConsole } from './components/console/consoleConfig'
function BrowseRedirect() {
const { '*': splat } = useParams()
return <Navigate to={`/app/${splat || ''}`} replace />
}
function Admin({ children }) {
return <RequireAdmin>{children}</RequireAdmin>
}
function Feature({ feature, children }) {
return <RequireFeature feature={feature}>{children}</RequireFeature>
}
const appChildren = [
{ index: true, element: <Home /> },
{ path: 'chat', element: <Chat /> },
{ path: 'chat/:model', element: <Chat /> },
{ path: 'image', element: <ImageGen /> },
{ path: 'image/:model', element: <ImageGen /> },
{ path: 'video', element: <VideoGen /> },
{ path: 'video/:model', element: <VideoGen /> },
{ path: '3d', element: <Feature feature="3d"><ThreeDGen /></Feature> },
{ path: '3d/:model', element: <Feature feature="3d"><ThreeDGen /></Feature> },
{ path: 'tts', element: <TTS /> },
{ path: 'tts/:model', element: <TTS /> },
{ path: 'sound', element: <Sound /> },
{ path: 'sound/:model', element: <Sound /> },
{ path: 'transform', element: <Feature feature="audio_transform"><AudioTransform /></Feature> },
{ path: 'transform/:model', element: <Feature feature="audio_transform"><AudioTransform /></Feature> },
{ path: 'studio', element: <Studio /> },
{ path: 'talk', element: <Talk /> },
{ path: 'account', element: <Account /> },
// Build console — Automation, Training, and Recognition groups share one rail.
// Only the section landing pages live under the rail; deep create/edit/chat
// flows below render full-width.
{
element: <ConsoleLayout config={buildConsole} />,
children: [
{ path: 'agents', element: <Feature feature="agents"><Agents /></Feature> },
{ path: 'skills', element: <Feature feature="skills"><Skills /></Feature> },
{ path: 'collections', element: <Feature feature="collections"><Collections /></Feature> },
{ path: 'agent-jobs', element: <Feature feature="mcp_jobs"><AgentJobs /></Feature> },
{ path: 'fine-tune', element: <Feature feature="fine_tuning"><FineTune /></Feature> },
{ path: 'quantize', element: <Feature feature="quantization"><Quantize /></Feature> },
{ path: 'face', element: <Feature feature="face_recognition"><FaceRecognition /></Feature> },
{ path: 'face/:model', element: <Feature feature="face_recognition"><FaceRecognition /></Feature> },
{ path: 'voice', element: <Feature feature="voice_recognition"><VoiceRecognition /></Feature> },
{ path: 'voice/:model', element: <Feature feature="voice_recognition"><VoiceRecognition /></Feature> },
],
},
// Build deep flows — full-width, no rail.
{ path: 'agents/new', element: <Feature feature="agents"><AgentCreate /></Feature> },
{ path: 'agents/:name/edit', element: <Feature feature="agents"><AgentCreate /></Feature> },
{ path: 'agents/:name/chat', element: <Feature feature="agents"><AgentChat /></Feature> },
{ path: 'agents/:name/status', element: <Feature feature="agents"><AgentStatus /></Feature> },
{ path: 'collections/:name', element: <Feature feature="collections"><CollectionDetails /></Feature> },
{ path: 'skills/new', element: <Feature feature="skills"><SkillEdit /></Feature> },
{ path: 'skills/edit/:name', element: <Feature feature="skills"><SkillEdit /></Feature> },
{ path: 'agent-jobs/tasks/new', element: <Feature feature="mcp_jobs"><AgentTaskDetails /></Feature> },
{ path: 'agent-jobs/tasks/:id', element: <Feature feature="mcp_jobs"><AgentTaskDetails /></Feature> },
{ path: 'agent-jobs/tasks/:id/edit', element: <Feature feature="mcp_jobs"><AgentTaskDetails /></Feature> },
{ path: 'agent-jobs/jobs/:id', element: <Feature feature="mcp_jobs"><AgentJobDetails /></Feature> },
// Operate console (admin).
{
element: <ConsoleLayout config={operateConsole} />,
children: [
{ path: 'backends', element: <Admin><Backends /></Admin> },
{ path: 'activity', element: <Admin><Activity /></Admin> },
{ path: 'voice-library', element: <Admin><VoiceLibrary /></Admin> },
{ path: 'settings', element: <Admin><Settings /></Admin> },
{ path: 'traces', element: <Admin><Traces /></Admin> },
{ path: 'backend-logs/:modelId', element: <Admin><BackendLogs /></Admin> },
{ path: 'p2p', element: <Admin><P2P /></Admin> },
{ path: 'nodes', element: <Admin><Nodes /></Admin> },
{ path: 'nodes/:id', element: <Admin><NodeDetail /></Admin> },
{ path: 'scheduling', element: <Admin><Scheduling /></Admin> },
{ path: 'node-backend-logs/:nodeId/:modelId', element: <Admin><NodeBackendLogs /></Admin> },
{ path: 'usage', element: <Usage /> },
{ path: 'users', element: <RequireAuthEnabled><Admin><Users /></Admin></RequireAuthEnabled> },
{ path: 'middleware', element: <Admin><Middleware /></Admin> },
{ path: 'manage', element: <Admin><Manage /></Admin> },
],
},
// Models management (Install Models) — top-level destination, full-width.
{ path: 'models', element: <Admin><Models /></Admin> },
{ path: 'voice-library/new', element: <Admin><VoiceProfileCreate /></Admin> },
{ path: 'model-editor', element: <Admin><ModelEditor /></Admin> },
{ path: 'model-editor/:name', element: <Admin><ModelEditor /></Admin> },
{ path: 'import-model', element: <Admin><ImportModel /></Admin> },
{ path: '*', element: <NotFound /> },
]
export const router = createBrowserRouter([
{
path: '/login',
element: <Login />,
},
{
path: '/invite/:code',
element: <Login />,
},
{
path: '/explorer',
element: <Explorer />,
},
{
path: '/app',
element: <RequireAuth><App /></RequireAuth>,
children: appChildren,
},
// Backward compatibility: redirect /browse/* to /app/*
{
path: '/browse/*',
element: <BrowseRedirect />,
},
{
path: '/',
element: <Navigate to="/app" replace />,
},
], { basename: routerBasename })