Files
LocalAI/core/http/react-ui/vite.config.js
Richard Palethorpe 9058a2bb46 feat: Add 3d generation UI/API and trellis2cpp backend (#10979)
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint

Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:

- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
  glb dst, seed/step/cfg_scale/texture_steps, quality and background
  enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
  the backend interfaces; connection-evicting and distributed-node
  wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
  backend), '3d' canonical usecase string mapped to the Generate3D
  method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
  OpenAIResponse with a /generated-3d URL or b64_json; conditioning
  image accepted as URL, base64, or data URI; quality/background
  validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend

Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:

- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
  asserted at startup), eager pipeline load with model-set validation
  (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
  exactly like the upstream demo), Generate3D via t2_generate +
  t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
  cover resolution/validation/param mapping — CI never downloads the
  multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
  libggml sonames collide across variants, unlike sd-ggml's flat
  renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
  amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
  latest/master image entries, bump_deps tracking of the pbr-textures
  branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
  llama-cpp so the .gguf match isn't stolen) and expands any trellis
  URI to the full 10-file component set spanning the three LocalAI-io
  HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
  trellis2-4b-geometry (512 untextured) with verified sha256s.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(ui): 3D generation page with native GLB viewer and IndexedDB history

Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:

- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
  renderer (quaternion trackball, metallic-roughness PBR, ACES,
  hidden-line wireframe with a bounded index budget) and pairs it with
  a minimal GLB parser for the two forms t2_bake_glb emits — dense
  vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
  normalized integers) and the opt-in UV-atlas textured form. Parsing
  happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
  the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
  GLBs are multi-MB binaries localStorage can't hold — and the page
  offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
  /api/models/capabilities serves), threeDApi, router entries, Studio
  tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
  one-triangle vertex-PBR GLB through the parser/viewer and exercises
  IndexedDB persistence, selection, deletion, and API errors.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(3d): address API correctness and UX issues

Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.

Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(3d): add previewable print remeshing

Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.

Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* build(trellis2cpp): centralize remesh dependency pins

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(kokoros): implement Generate3D stub for new proto RPC

The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.

Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
2026-07-29 16:15:04 +02:00

85 lines
3.4 KiB
JavaScript

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import istanbul from 'vite-plugin-istanbul'
const backendUrl = process.env.LOCALAI_URL || 'http://localhost:8080'
// COVERAGE=true produces an instrumented build whose modules report istanbul
// counters on window.__coverage__, harvested by the Playwright coverage
// fixture (e2e/coverage-fixtures.js). Off by default so normal/dev/prod builds
// carry no instrumentation overhead.
const coverage = process.env.COVERAGE === 'true'
// COVERAGE_V8=true produces a NON-instrumented build with source maps, so the
// Playwright coverage fixture can collect Chromium V8 coverage (near-zero
// runtime overhead, unlike istanbul's build-time counters) and map it back to
// source via v8-to-istanbul. Mutually exclusive with COVERAGE.
const coverageV8 = process.env.COVERAGE_V8 === 'true'
export default defineConfig({
plugins: [
react(),
...(coverage
? [
istanbul({
include: 'src/**/*',
extension: ['.js', '.jsx', '.ts', '.tsx'],
requireEnv: false,
// The e2e suite runs against `vite build` output, not the dev
// server, so instrumentation must be applied to the production
// build too (the plugin only instruments dev mode otherwise).
forceBuildInstrument: true,
}),
]
: []),
],
// Relative base so every generated URL (entry scripts in index.html, CSS
// `url()` font references, and lazily-imported route chunks) resolves against
// the file that references it rather than the origin root. When LocalAI is
// served under a reverse-proxy subpath (X-Forwarded-Prefix, e.g. `/llm/`),
// an absolute `/assets/...` bypasses the prefix and 404s — breaking fonts
// ("tofu" glyphs) and lazy-loaded chunks. index.html's now-relative refs
// resolve via the `<base href>` that serveIndex always injects (see
// core/http/app.go), so both proxied and root deployments load correctly.
base: './',
server: {
port: 3000,
proxy: {
'/api': backendUrl,
'/v1': backendUrl,
'/tts': backendUrl,
'/video': backendUrl,
'/backend': backendUrl,
'/models': backendUrl,
'/backends': backendUrl,
'/swagger': backendUrl,
'/static': backendUrl,
'/generated-audio': backendUrl,
'/generated-images': backendUrl,
'/generated-videos': backendUrl,
'/generated-3d': backendUrl,
'/3d': backendUrl,
'/version': backendUrl,
'/system': backendUrl,
},
},
build: {
outDir: 'dist',
assetsDir: 'assets',
// Source maps are needed only to map V8 coverage back to original sources.
sourcemap: coverageV8,
rollupOptions: {
output: {
// The coverage build inlines all dynamic imports into a single chunk.
// The app is route-code-split (router.jsx uses React.lazy), so a normal
// build emits ~50 lazy chunks. V8 coverage only sees chunks a test
// actually loaded, so untested pages would silently drop out of the
// denominator and inflate the percentage. Bundling everything into one
// chunk for the coverage build keeps the denominator complete and the
// measurement invariant to how production is split. Production builds
// (COVERAGE_V8 unset) keep code-splitting for fast first paint.
inlineDynamicImports: coverageV8,
},
},
},
})