mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 01:48:06 -04:00
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint Adds the plumbing for image-conditioned 3D asset generation (binary glTF / GLB output), modeled on the video generation path: - backend.proto: Generate3D RPC + Generate3DRequest (staged image src, glb dst, seed/step/cfg_scale/texture_steps, quality and background enums, params map for backend-specific extras) - pkg/grpc: thread Generate3D through client, server, embed, base and the backend interfaces; connection-evicting and distributed-node wrappers (in-flight tracking + file staging) included - core/config: FLAG_3D usecase (guessed only for the trellis2cpp backend), '3d' canonical usecase string mapped to the Generate3D method, and a '3d' output modality - REST: POST /v1/3d/generations (+ unversioned alias) returning OpenAIResponse with a /generated-3d URL or b64_json; conditioning image accepted as URL, base64, or data URI; quality/background validated at the edge; .glb served as model/gltf-binary - auth: '3d' route feature (default ON); /api/instructions entry Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(trellis2cpp): add the trellis2.cpp image-to-3D backend Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2, pbr-textures branch) as a Go+purego backend, following the stablediffusion-ggml pattern: - backend/go/trellis2cpp: purego bindings to the flat C ABI (v9, asserted at startup), eager pipeline load with model-set validation (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured exactly like the upstream demo), Generate3D via t2_generate + t2_bake_glb writing a binary glTF to dst. Weight-free unit tests cover resolution/validation/param mapping — CI never downloads the multi-GB GGUF set or runs inference. - CPU SIMD variants build into per-variant directories (the shared libggml sonames collide across variants, unlike sd-ggml's flat renamed-.so scheme); run.sh picks one via /proc/cpuinfo. - CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta + latest/master image entries, bump_deps tracking of the pbr-textures branch, changed-backends.js mapping, top-level Makefile targets. - Importer: auto-detects trellis GGUF repos/URIs (registered before llama-cpp so the .gguf match isn't stolen) and expands any trellis URI to the full 10-file component set spanning the three LocalAI-io HF repos. - Gallery: trellis2-4b (full PBR + 1024 cascade) and trellis2-4b-geometry (512 untextured) with verified sha256s. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(ui): 3D generation page with native GLB viewer and IndexedDB history Adds a Studio tab + /app/3d page for the new image-to-3D endpoint: - GlbViewer ports the trellis2cpp demo's dependency-free WebGL2 renderer (quaternion trackball, metallic-roughness PBR, ACES, hidden-line wireframe with a bounded index budget) and pairs it with a minimal GLB parser for the two forms t2_bake_glb emits — dense vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as normalized integers) and the opt-in UV-atlas textured form. Parsing happens before any GL so stats and errors render without WebGL2. - use3DHistory stores past generations (params, input thumbnail, and the GLB blob itself) in IndexedDB with keep-newest-20 eviction — GLBs are multi-MB binaries localStorage can't hold — and the page offers a download button for the active GLB. - Wiring: CAP_3D capability constant (FLAG_3D — the exact string /api/models/capabilities serves), threeDApi, router entries, Studio tab, vite dev proxy, en locale keys. - e2e: render-smoke entry plus a focused spec that feeds a real one-triangle vertex-PBR GLB through the parser/viewer and exercises IndexedDB persistence, selection, deletion, and API errors. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(3d): address API correctness and UX issues Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it. Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(3d): add previewable print remeshing Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download. Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * build(trellis2cpp): centralize remesh dependency pins Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(kokoros): implement Generate3D stub for new proto RPC The Generate3D RPC added to backend.proto for the trellis2cpp backend made tonic's generated Backend trait require generate3_d, breaking the kokoros-grpc build. Return unimplemented like the other unsupported modalities. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com> Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
85 lines
3.4 KiB
JavaScript
85 lines
3.4 KiB
JavaScript
import { defineConfig } from 'vite'
|
|
import react from '@vitejs/plugin-react'
|
|
import istanbul from 'vite-plugin-istanbul'
|
|
|
|
const backendUrl = process.env.LOCALAI_URL || 'http://localhost:8080'
|
|
|
|
// COVERAGE=true produces an instrumented build whose modules report istanbul
|
|
// counters on window.__coverage__, harvested by the Playwright coverage
|
|
// fixture (e2e/coverage-fixtures.js). Off by default so normal/dev/prod builds
|
|
// carry no instrumentation overhead.
|
|
const coverage = process.env.COVERAGE === 'true'
|
|
// COVERAGE_V8=true produces a NON-instrumented build with source maps, so the
|
|
// Playwright coverage fixture can collect Chromium V8 coverage (near-zero
|
|
// runtime overhead, unlike istanbul's build-time counters) and map it back to
|
|
// source via v8-to-istanbul. Mutually exclusive with COVERAGE.
|
|
const coverageV8 = process.env.COVERAGE_V8 === 'true'
|
|
|
|
export default defineConfig({
|
|
plugins: [
|
|
react(),
|
|
...(coverage
|
|
? [
|
|
istanbul({
|
|
include: 'src/**/*',
|
|
extension: ['.js', '.jsx', '.ts', '.tsx'],
|
|
requireEnv: false,
|
|
// The e2e suite runs against `vite build` output, not the dev
|
|
// server, so instrumentation must be applied to the production
|
|
// build too (the plugin only instruments dev mode otherwise).
|
|
forceBuildInstrument: true,
|
|
}),
|
|
]
|
|
: []),
|
|
],
|
|
// Relative base so every generated URL (entry scripts in index.html, CSS
|
|
// `url()` font references, and lazily-imported route chunks) resolves against
|
|
// the file that references it rather than the origin root. When LocalAI is
|
|
// served under a reverse-proxy subpath (X-Forwarded-Prefix, e.g. `/llm/`),
|
|
// an absolute `/assets/...` bypasses the prefix and 404s — breaking fonts
|
|
// ("tofu" glyphs) and lazy-loaded chunks. index.html's now-relative refs
|
|
// resolve via the `<base href>` that serveIndex always injects (see
|
|
// core/http/app.go), so both proxied and root deployments load correctly.
|
|
base: './',
|
|
server: {
|
|
port: 3000,
|
|
proxy: {
|
|
'/api': backendUrl,
|
|
'/v1': backendUrl,
|
|
'/tts': backendUrl,
|
|
'/video': backendUrl,
|
|
'/backend': backendUrl,
|
|
'/models': backendUrl,
|
|
'/backends': backendUrl,
|
|
'/swagger': backendUrl,
|
|
'/static': backendUrl,
|
|
'/generated-audio': backendUrl,
|
|
'/generated-images': backendUrl,
|
|
'/generated-videos': backendUrl,
|
|
'/generated-3d': backendUrl,
|
|
'/3d': backendUrl,
|
|
'/version': backendUrl,
|
|
'/system': backendUrl,
|
|
},
|
|
},
|
|
build: {
|
|
outDir: 'dist',
|
|
assetsDir: 'assets',
|
|
// Source maps are needed only to map V8 coverage back to original sources.
|
|
sourcemap: coverageV8,
|
|
rollupOptions: {
|
|
output: {
|
|
// The coverage build inlines all dynamic imports into a single chunk.
|
|
// The app is route-code-split (router.jsx uses React.lazy), so a normal
|
|
// build emits ~50 lazy chunks. V8 coverage only sees chunks a test
|
|
// actually loaded, so untested pages would silently drop out of the
|
|
// denominator and inflate the percentage. Bundling everything into one
|
|
// chunk for the coverage build keeps the denominator complete and the
|
|
// measurement invariant to how production is split. Production builds
|
|
// (COVERAGE_V8 unset) keep code-splitting for fast first paint.
|
|
inlineDynamicImports: coverageV8,
|
|
},
|
|
},
|
|
},
|
|
})
|