* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint
Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:
- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
glb dst, seed/step/cfg_scale/texture_steps, quality and background
enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
the backend interfaces; connection-evicting and distributed-node
wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
backend), '3d' canonical usecase string mapped to the Generate3D
method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
OpenAIResponse with a /generated-3d URL or b64_json; conditioning
image accepted as URL, base64, or data URI; quality/background
validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend
Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:
- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
asserted at startup), eager pipeline load with model-set validation
(refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
exactly like the upstream demo), Generate3D via t2_generate +
t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
cover resolution/validation/param mapping — CI never downloads the
multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
libggml sonames collide across variants, unlike sd-ggml's flat
renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
latest/master image entries, bump_deps tracking of the pbr-textures
branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
llama-cpp so the .gguf match isn't stolen) and expands any trellis
URI to the full 10-file component set spanning the three LocalAI-io
HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
trellis2-4b-geometry (512 untextured) with verified sha256s.
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(ui): 3D generation page with native GLB viewer and IndexedDB history
Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:
- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
renderer (quaternion trackball, metallic-roughness PBR, ACES,
hidden-line wireframe with a bounded index budget) and pairs it with
a minimal GLB parser for the two forms t2_bake_glb emits — dense
vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
normalized integers) and the opt-in UV-atlas textured form. Parsing
happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
GLBs are multi-MB binaries localStorage can't hold — and the page
offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
/api/models/capabilities serves), threeDApi, router entries, Studio
tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
one-triangle vertex-PBR GLB through the parser/viewer and exercises
IndexedDB persistence, selection, deletion, and API errors.
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(3d): address API correctness and UX issues
Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.
Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(3d): add previewable print remeshing
Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.
Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* build(trellis2cpp): centralize remesh dependency pins
Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(kokoros): implement Generate3D stub for new proto RPC
The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
---------
Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
The Vite build emitted path-absolute asset URLs (base: '/'). index.html
entry scripts and the favicon were rewritten to include the reverse-proxy
prefix in serveIndex, but two reference kinds are not in index.html and so
bypassed that rewrite:
- CSS `url()` font references (e.g. Font Awesome .woff2), which the browser
resolves relative to the stylesheet and which `<base href>` never affects
- lazily-imported route chunks, whose preload base came from the absolute
Vite base
Under a subpath mount (X-Forwarded-Prefix: /llm/) both were fetched from the
origin root, 404ing — missing-glyph "tofu" icons and broken lazy-loaded pages.
Switch Vite to a relative base ('./') so every generated URL resolves against
the file that references it: CSS fonts and route chunks now load from
`/llm/assets/...`, and index.html's now-relative entry refs resolve via the
`<base href>` serveIndex already injects on every response. Root deployments
are unaffected. The existing path-absolute rewrite in app.go still covers the
public `/favicon.svg`.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* Curate the highlight.js build to ~29 languages (lib/core + the
common set) instead of the full ~190-grammar default: -787 KB raw /
-230 KB gz on the base bundle.
* Code-split every route via React.lazy with a per-layout <Suspense>
in App.jsx so the sidebar stays mounted on navigation. Initial entry
chunk drops from 3194 KB raw / 887 KB gz to 397 KB / 122 KB (-87%).
Warm chunks on sidebar hover/focus/touch via a preload registry so
the click finds the chunk already in flight or cached.
* Migrate Playwright coverage from istanbul (build-time counters) to
native Chromium V8 coverage, with per-worker accumulation +
conversion. Suite drops from 71s to 30s at 20 workers (~58%) at the
non-instrumented floor.
* Keep the coverage gate bundling-invariant: the coverage build inlines
dynamic imports so every shipped source file lands in the denominator
(otherwise untested page chunks silently drop out and inflate the
percentage). Production builds stay code-split.
* Add UI_TEST_WORKERS=N Makefile knob; tighten coverage tolerance to
0.8pp now that jitter sits near istanbul's ~0.5pp again.
Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
- Strict monotonic Go coverage gate (make test-coverage-check, 45% baseline)
run in CI; fixes ginkgo dropping all-but-one coverprofile across multiple
recursive roots, builds with -tags auth, and folds in the in-process
tests/e2e suite via --coverpkg.
- React UI e2e coverage (make test-ui-coverage: vite-plugin-istanbul + nyc,
nix-provided Chromium) plus e2e specs for 6 previously-untested pages, and a
UI coverage gate (make test-ui-coverage-check) with a small tolerance since
e2e line coverage jitters ~0.5pp run-to-run.
- pre-commit hook: lint + coverage on Go changes, Playwright e2e + UI coverage
gate on react-ui changes; install with make install-hooks.
- New Go handler tests (settings, branding), hermetic base64 download test.
- fix(ui): model editor reads vram_display (snake_case), so the VRAM estimate
renders again; covered by a regression test.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Richard Palethorpe <io@richiejp.com>
