Files
LocalAI/core/http/auth/permissions.go
Richard Palethorpe 9058a2bb46 feat: Add 3d generation UI/API and trellis2cpp backend (#10979)
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint

Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:

- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
  glb dst, seed/step/cfg_scale/texture_steps, quality and background
  enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
  the backend interfaces; connection-evicting and distributed-node
  wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
  backend), '3d' canonical usecase string mapped to the Generate3D
  method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
  OpenAIResponse with a /generated-3d URL or b64_json; conditioning
  image accepted as URL, base64, or data URI; quality/background
  validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend

Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:

- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
  asserted at startup), eager pipeline load with model-set validation
  (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
  exactly like the upstream demo), Generate3D via t2_generate +
  t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
  cover resolution/validation/param mapping — CI never downloads the
  multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
  libggml sonames collide across variants, unlike sd-ggml's flat
  renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
  amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
  latest/master image entries, bump_deps tracking of the pbr-textures
  branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
  llama-cpp so the .gguf match isn't stolen) and expands any trellis
  URI to the full 10-file component set spanning the three LocalAI-io
  HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
  trellis2-4b-geometry (512 untextured) with verified sha256s.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(ui): 3D generation page with native GLB viewer and IndexedDB history

Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:

- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
  renderer (quaternion trackball, metallic-roughness PBR, ACES,
  hidden-line wireframe with a bounded index budget) and pairs it with
  a minimal GLB parser for the two forms t2_bake_glb emits — dense
  vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
  normalized integers) and the opt-in UV-atlas textured form. Parsing
  happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
  the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
  GLBs are multi-MB binaries localStorage can't hold — and the page
  offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
  /api/models/capabilities serves), threeDApi, router entries, Studio
  tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
  one-triangle vertex-PBR GLB through the parser/viewer and exercises
  IndexedDB persistence, selection, deletion, and API errors.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(3d): address API correctness and UX issues

Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.

Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(3d): add previewable print remeshing

Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.

Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* build(trellis2cpp): centralize remesh dependency pins

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(kokoros): implement Generate3D stub for new proto RPC

The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.

Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
2026-07-29 16:15:04 +02:00

233 lines
7.2 KiB
Go

package auth
import (
"github.com/google/uuid"
"github.com/labstack/echo/v4"
"gorm.io/gorm"
)
const contextKeyPermissions = "auth_permissions"
// GetCachedUserPermissions returns the user's permission record, using a
// request-scoped cache stored in the echo context. This avoids duplicate
// DB lookups when multiple middlewares (RequireRouteFeature, RequireModelAccess)
// both need permissions in the same request.
func GetCachedUserPermissions(c echo.Context, db *gorm.DB, userID string) (*UserPermission, error) {
if perm, ok := c.Get(contextKeyPermissions).(*UserPermission); ok && perm != nil {
return perm, nil
}
perm, err := GetUserPermissions(db, userID)
if err != nil {
return nil, err
}
c.Set(contextKeyPermissions, perm)
return perm, nil
}
// Feature name constants — all code must use these, never bare strings.
const (
// Agent features (default OFF for new users)
FeatureAgents = "agents"
FeatureSkills = "skills"
FeatureCollections = "collections"
FeatureMCPJobs = "mcp_jobs"
FeatureLocalAIAssistant = "localai_assistant"
// General features (default OFF for new users)
FeatureFineTuning = "fine_tuning"
FeatureQuantization = "quantization"
// API features (default ON for new users)
FeatureChat = "chat"
FeatureImages = "images"
FeatureAudioSpeech = "audio_speech"
FeatureAudioTranscription = "audio_transcription"
FeatureAudioDiarization = "audio_diarization"
FeatureAudioClassification = "audio_classification"
FeatureVAD = "vad"
FeatureDetection = "detection"
FeatureVideo = "video"
Feature3D = "3d"
FeatureEmbeddings = "embeddings"
FeatureSound = "sound"
FeatureRealtime = "realtime"
FeatureRerank = "rerank"
FeatureTokenize = "tokenize"
FeatureMCP = "mcp"
FeatureStores = "stores"
FeatureFaceRecognition = "face_recognition"
FeatureVoiceRecognition = "voice_recognition"
FeatureAudioTransform = "audio_transform"
// FeaturePIIFilter gates the synchronous PII analyze/redact service
// (POST /api/pii/{analyze,redact}). Default ON like the other API
// features; the admin-only events log is gated separately in-handler.
FeaturePIIFilter = "pii_filter"
)
// AgentFeatures lists agent-related features (default OFF).
var AgentFeatures = []string{FeatureAgents, FeatureSkills, FeatureCollections, FeatureMCPJobs, FeatureLocalAIAssistant}
// GeneralFeatures lists general features (default OFF).
var GeneralFeatures = []string{FeatureFineTuning, FeatureQuantization}
// APIFeatures lists API endpoint features (default ON).
var APIFeatures = []string{
FeatureChat, FeatureImages, FeatureAudioSpeech, FeatureAudioTranscription,
FeatureAudioDiarization, FeatureAudioClassification,
FeatureVAD, FeatureDetection, FeatureVideo, Feature3D, FeatureEmbeddings, FeatureSound,
FeatureRealtime, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
FeatureFaceRecognition, FeatureVoiceRecognition, FeatureAudioTransform,
FeaturePIIFilter,
}
// AllFeatures lists all known features (used by UI and validation).
var AllFeatures = append(append(append([]string{}, AgentFeatures...), GeneralFeatures...), APIFeatures...)
// defaultOnFeatures is the set of features that default to ON when absent from a user's permission map.
var defaultOnFeatures = func() map[string]bool {
m := map[string]bool{}
for _, f := range APIFeatures {
m[f] = true
}
return m
}()
// isDefaultOnFeature returns true if the feature defaults to ON when not explicitly set.
func isDefaultOnFeature(feature string) bool {
return defaultOnFeatures[feature]
}
// GetUserPermissions returns the permission record for a user, creating a default
// (empty map = all disabled) if none exists.
func GetUserPermissions(db *gorm.DB, userID string) (*UserPermission, error) {
var perm UserPermission
err := db.Where("user_id = ?", userID).First(&perm).Error
if err == gorm.ErrRecordNotFound {
perm = UserPermission{
ID: uuid.New().String(),
UserID: userID,
Permissions: PermissionMap{},
}
if err := db.Create(&perm).Error; err != nil {
return nil, err
}
return &perm, nil
}
if err != nil {
return nil, err
}
return &perm, nil
}
// UpdateUserPermissions upserts the permission map for a user.
func UpdateUserPermissions(db *gorm.DB, userID string, perms PermissionMap) error {
var perm UserPermission
err := db.Where("user_id = ?", userID).First(&perm).Error
if err == gorm.ErrRecordNotFound {
perm = UserPermission{
ID: uuid.New().String(),
UserID: userID,
Permissions: perms,
}
return db.Create(&perm).Error
}
if err != nil {
return err
}
perm.Permissions = perms
return db.Save(&perm).Error
}
// HasFeatureAccess returns true if the user is an admin or has the given feature enabled.
// When a feature key is absent from the user's permission map, it checks whether the
// feature defaults to ON (API features) or OFF (agent features) for backward compatibility.
func HasFeatureAccess(db *gorm.DB, user *User, feature string) bool {
if user == nil {
return false
}
if user.Role == RoleAdmin {
return true
}
perm, err := GetUserPermissions(db, user.ID)
if err != nil {
return false
}
val, exists := perm.Permissions[feature]
if !exists {
return isDefaultOnFeature(feature)
}
return val
}
// GetPermissionMapForUser returns the effective permission map for a user.
// Admins get all features as true (virtual).
// For regular users, absent keys are filled with their defaults so the
// UI/API always returns a complete picture.
func GetPermissionMapForUser(db *gorm.DB, user *User) PermissionMap {
if user == nil {
return PermissionMap{}
}
if user.Role == RoleAdmin {
m := PermissionMap{}
for _, f := range AllFeatures {
m[f] = true
}
return m
}
perm, err := GetUserPermissions(db, user.ID)
if err != nil {
return PermissionMap{}
}
// Fill in defaults for absent keys
effective := PermissionMap{}
for _, f := range AllFeatures {
val, exists := perm.Permissions[f]
if exists {
effective[f] = val
} else {
effective[f] = isDefaultOnFeature(f)
}
}
return effective
}
// GetModelAllowlist returns the model allowlist for a user.
func GetModelAllowlist(db *gorm.DB, userID string) ModelAllowlist {
perm, err := GetUserPermissions(db, userID)
if err != nil {
return ModelAllowlist{}
}
return perm.AllowedModels
}
// UpdateModelAllowlist updates the model allowlist for a user.
func UpdateModelAllowlist(db *gorm.DB, userID string, allowlist ModelAllowlist) error {
perm, err := GetUserPermissions(db, userID)
if err != nil {
return err
}
perm.AllowedModels = allowlist
return db.Save(perm).Error
}
// IsModelAllowed returns true if the user is allowed to use the given model.
// Admins always have access. If the allowlist is not enabled, all models are allowed.
func IsModelAllowed(db *gorm.DB, user *User, modelName string) bool {
if user == nil {
return false
}
if user.Role == RoleAdmin {
return true
}
allowlist := GetModelAllowlist(db, user.ID)
if !allowlist.Enabled {
return true
}
for _, m := range allowlist.Models {
if m == modelName {
return true
}
}
return false
}