* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint
Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:
- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
glb dst, seed/step/cfg_scale/texture_steps, quality and background
enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
the backend interfaces; connection-evicting and distributed-node
wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
backend), '3d' canonical usecase string mapped to the Generate3D
method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
OpenAIResponse with a /generated-3d URL or b64_json; conditioning
image accepted as URL, base64, or data URI; quality/background
validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend
Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:
- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
asserted at startup), eager pipeline load with model-set validation
(refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
exactly like the upstream demo), Generate3D via t2_generate +
t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
cover resolution/validation/param mapping — CI never downloads the
multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
libggml sonames collide across variants, unlike sd-ggml's flat
renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
latest/master image entries, bump_deps tracking of the pbr-textures
branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
llama-cpp so the .gguf match isn't stolen) and expands any trellis
URI to the full 10-file component set spanning the three LocalAI-io
HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
trellis2-4b-geometry (512 untextured) with verified sha256s.
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(ui): 3D generation page with native GLB viewer and IndexedDB history
Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:
- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
renderer (quaternion trackball, metallic-roughness PBR, ACES,
hidden-line wireframe with a bounded index budget) and pairs it with
a minimal GLB parser for the two forms t2_bake_glb emits — dense
vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
normalized integers) and the opt-in UV-atlas textured form. Parsing
happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
GLBs are multi-MB binaries localStorage can't hold — and the page
offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
/api/models/capabilities serves), threeDApi, router entries, Studio
tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
one-triangle vertex-PBR GLB through the parser/viewer and exercises
IndexedDB persistence, selection, deletion, and API errors.
Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(3d): address API correctness and UX issues
Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.
Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(3d): add previewable print remeshing
Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.
Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* build(trellis2cpp): centralize remesh dependency pins
Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(kokoros): implement Generate3D stub for new proto RPC
The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
---------
Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
