mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint Adds the plumbing for image-conditioned 3D asset generation (binary glTF / GLB output), modeled on the video generation path: - backend.proto: Generate3D RPC + Generate3DRequest (staged image src, glb dst, seed/step/cfg_scale/texture_steps, quality and background enums, params map for backend-specific extras) - pkg/grpc: thread Generate3D through client, server, embed, base and the backend interfaces; connection-evicting and distributed-node wrappers (in-flight tracking + file staging) included - core/config: FLAG_3D usecase (guessed only for the trellis2cpp backend), '3d' canonical usecase string mapped to the Generate3D method, and a '3d' output modality - REST: POST /v1/3d/generations (+ unversioned alias) returning OpenAIResponse with a /generated-3d URL or b64_json; conditioning image accepted as URL, base64, or data URI; quality/background validated at the edge; .glb served as model/gltf-binary - auth: '3d' route feature (default ON); /api/instructions entry Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(trellis2cpp): add the trellis2.cpp image-to-3D backend Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2, pbr-textures branch) as a Go+purego backend, following the stablediffusion-ggml pattern: - backend/go/trellis2cpp: purego bindings to the flat C ABI (v9, asserted at startup), eager pipeline load with model-set validation (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured exactly like the upstream demo), Generate3D via t2_generate + t2_bake_glb writing a binary glTF to dst. Weight-free unit tests cover resolution/validation/param mapping — CI never downloads the multi-GB GGUF set or runs inference. - CPU SIMD variants build into per-variant directories (the shared libggml sonames collide across variants, unlike sd-ggml's flat renamed-.so scheme); run.sh picks one via /proc/cpuinfo. - CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta + latest/master image entries, bump_deps tracking of the pbr-textures branch, changed-backends.js mapping, top-level Makefile targets. - Importer: auto-detects trellis GGUF repos/URIs (registered before llama-cpp so the .gguf match isn't stolen) and expands any trellis URI to the full 10-file component set spanning the three LocalAI-io HF repos. - Gallery: trellis2-4b (full PBR + 1024 cascade) and trellis2-4b-geometry (512 untextured) with verified sha256s. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(ui): 3D generation page with native GLB viewer and IndexedDB history Adds a Studio tab + /app/3d page for the new image-to-3D endpoint: - GlbViewer ports the trellis2cpp demo's dependency-free WebGL2 renderer (quaternion trackball, metallic-roughness PBR, ACES, hidden-line wireframe with a bounded index budget) and pairs it with a minimal GLB parser for the two forms t2_bake_glb emits — dense vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as normalized integers) and the opt-in UV-atlas textured form. Parsing happens before any GL so stats and errors render without WebGL2. - use3DHistory stores past generations (params, input thumbnail, and the GLB blob itself) in IndexedDB with keep-newest-20 eviction — GLBs are multi-MB binaries localStorage can't hold — and the page offers a download button for the active GLB. - Wiring: CAP_3D capability constant (FLAG_3D — the exact string /api/models/capabilities serves), threeDApi, router entries, Studio tab, vite dev proxy, en locale keys. - e2e: render-smoke entry plus a focused spec that feeds a real one-triangle vertex-PBR GLB through the parser/viewer and exercises IndexedDB persistence, selection, deletion, and API errors. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(3d): address API correctness and UX issues Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it. Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(3d): add previewable print remeshing Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download. Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * build(trellis2cpp): centralize remesh dependency pins Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(kokoros): implement Generate3D stub for new proto RPC The Generate3D RPC added to backend.proto for the trellis2cpp backend made tonic's generated Backend trait require generate3_d, breaking the kokoros-grpc build. Return unimplemented like the other unsupported modalities. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com> Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
233 lines
7.2 KiB
Go
233 lines
7.2 KiB
Go
package auth
|
|
|
|
import (
|
|
"github.com/google/uuid"
|
|
"github.com/labstack/echo/v4"
|
|
"gorm.io/gorm"
|
|
)
|
|
|
|
const contextKeyPermissions = "auth_permissions"
|
|
|
|
// GetCachedUserPermissions returns the user's permission record, using a
|
|
// request-scoped cache stored in the echo context. This avoids duplicate
|
|
// DB lookups when multiple middlewares (RequireRouteFeature, RequireModelAccess)
|
|
// both need permissions in the same request.
|
|
func GetCachedUserPermissions(c echo.Context, db *gorm.DB, userID string) (*UserPermission, error) {
|
|
if perm, ok := c.Get(contextKeyPermissions).(*UserPermission); ok && perm != nil {
|
|
return perm, nil
|
|
}
|
|
perm, err := GetUserPermissions(db, userID)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
c.Set(contextKeyPermissions, perm)
|
|
return perm, nil
|
|
}
|
|
|
|
// Feature name constants — all code must use these, never bare strings.
|
|
const (
|
|
// Agent features (default OFF for new users)
|
|
FeatureAgents = "agents"
|
|
FeatureSkills = "skills"
|
|
FeatureCollections = "collections"
|
|
FeatureMCPJobs = "mcp_jobs"
|
|
FeatureLocalAIAssistant = "localai_assistant"
|
|
|
|
// General features (default OFF for new users)
|
|
FeatureFineTuning = "fine_tuning"
|
|
FeatureQuantization = "quantization"
|
|
|
|
// API features (default ON for new users)
|
|
FeatureChat = "chat"
|
|
FeatureImages = "images"
|
|
FeatureAudioSpeech = "audio_speech"
|
|
FeatureAudioTranscription = "audio_transcription"
|
|
FeatureAudioDiarization = "audio_diarization"
|
|
FeatureAudioClassification = "audio_classification"
|
|
FeatureVAD = "vad"
|
|
FeatureDetection = "detection"
|
|
FeatureVideo = "video"
|
|
Feature3D = "3d"
|
|
FeatureEmbeddings = "embeddings"
|
|
FeatureSound = "sound"
|
|
FeatureRealtime = "realtime"
|
|
FeatureRerank = "rerank"
|
|
FeatureTokenize = "tokenize"
|
|
FeatureMCP = "mcp"
|
|
FeatureStores = "stores"
|
|
FeatureFaceRecognition = "face_recognition"
|
|
FeatureVoiceRecognition = "voice_recognition"
|
|
FeatureAudioTransform = "audio_transform"
|
|
// FeaturePIIFilter gates the synchronous PII analyze/redact service
|
|
// (POST /api/pii/{analyze,redact}). Default ON like the other API
|
|
// features; the admin-only events log is gated separately in-handler.
|
|
FeaturePIIFilter = "pii_filter"
|
|
)
|
|
|
|
// AgentFeatures lists agent-related features (default OFF).
|
|
var AgentFeatures = []string{FeatureAgents, FeatureSkills, FeatureCollections, FeatureMCPJobs, FeatureLocalAIAssistant}
|
|
|
|
// GeneralFeatures lists general features (default OFF).
|
|
var GeneralFeatures = []string{FeatureFineTuning, FeatureQuantization}
|
|
|
|
// APIFeatures lists API endpoint features (default ON).
|
|
var APIFeatures = []string{
|
|
FeatureChat, FeatureImages, FeatureAudioSpeech, FeatureAudioTranscription,
|
|
FeatureAudioDiarization, FeatureAudioClassification,
|
|
FeatureVAD, FeatureDetection, FeatureVideo, Feature3D, FeatureEmbeddings, FeatureSound,
|
|
FeatureRealtime, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
|
|
FeatureFaceRecognition, FeatureVoiceRecognition, FeatureAudioTransform,
|
|
FeaturePIIFilter,
|
|
}
|
|
|
|
// AllFeatures lists all known features (used by UI and validation).
|
|
var AllFeatures = append(append(append([]string{}, AgentFeatures...), GeneralFeatures...), APIFeatures...)
|
|
|
|
// defaultOnFeatures is the set of features that default to ON when absent from a user's permission map.
|
|
var defaultOnFeatures = func() map[string]bool {
|
|
m := map[string]bool{}
|
|
for _, f := range APIFeatures {
|
|
m[f] = true
|
|
}
|
|
return m
|
|
}()
|
|
|
|
// isDefaultOnFeature returns true if the feature defaults to ON when not explicitly set.
|
|
func isDefaultOnFeature(feature string) bool {
|
|
return defaultOnFeatures[feature]
|
|
}
|
|
|
|
// GetUserPermissions returns the permission record for a user, creating a default
|
|
// (empty map = all disabled) if none exists.
|
|
func GetUserPermissions(db *gorm.DB, userID string) (*UserPermission, error) {
|
|
var perm UserPermission
|
|
err := db.Where("user_id = ?", userID).First(&perm).Error
|
|
if err == gorm.ErrRecordNotFound {
|
|
perm = UserPermission{
|
|
ID: uuid.New().String(),
|
|
UserID: userID,
|
|
Permissions: PermissionMap{},
|
|
}
|
|
if err := db.Create(&perm).Error; err != nil {
|
|
return nil, err
|
|
}
|
|
return &perm, nil
|
|
}
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &perm, nil
|
|
}
|
|
|
|
// UpdateUserPermissions upserts the permission map for a user.
|
|
func UpdateUserPermissions(db *gorm.DB, userID string, perms PermissionMap) error {
|
|
var perm UserPermission
|
|
err := db.Where("user_id = ?", userID).First(&perm).Error
|
|
if err == gorm.ErrRecordNotFound {
|
|
perm = UserPermission{
|
|
ID: uuid.New().String(),
|
|
UserID: userID,
|
|
Permissions: perms,
|
|
}
|
|
return db.Create(&perm).Error
|
|
}
|
|
if err != nil {
|
|
return err
|
|
}
|
|
perm.Permissions = perms
|
|
return db.Save(&perm).Error
|
|
}
|
|
|
|
// HasFeatureAccess returns true if the user is an admin or has the given feature enabled.
|
|
// When a feature key is absent from the user's permission map, it checks whether the
|
|
// feature defaults to ON (API features) or OFF (agent features) for backward compatibility.
|
|
func HasFeatureAccess(db *gorm.DB, user *User, feature string) bool {
|
|
if user == nil {
|
|
return false
|
|
}
|
|
if user.Role == RoleAdmin {
|
|
return true
|
|
}
|
|
perm, err := GetUserPermissions(db, user.ID)
|
|
if err != nil {
|
|
return false
|
|
}
|
|
val, exists := perm.Permissions[feature]
|
|
if !exists {
|
|
return isDefaultOnFeature(feature)
|
|
}
|
|
return val
|
|
}
|
|
|
|
// GetPermissionMapForUser returns the effective permission map for a user.
|
|
// Admins get all features as true (virtual).
|
|
// For regular users, absent keys are filled with their defaults so the
|
|
// UI/API always returns a complete picture.
|
|
func GetPermissionMapForUser(db *gorm.DB, user *User) PermissionMap {
|
|
if user == nil {
|
|
return PermissionMap{}
|
|
}
|
|
if user.Role == RoleAdmin {
|
|
m := PermissionMap{}
|
|
for _, f := range AllFeatures {
|
|
m[f] = true
|
|
}
|
|
return m
|
|
}
|
|
perm, err := GetUserPermissions(db, user.ID)
|
|
if err != nil {
|
|
return PermissionMap{}
|
|
}
|
|
// Fill in defaults for absent keys
|
|
effective := PermissionMap{}
|
|
for _, f := range AllFeatures {
|
|
val, exists := perm.Permissions[f]
|
|
if exists {
|
|
effective[f] = val
|
|
} else {
|
|
effective[f] = isDefaultOnFeature(f)
|
|
}
|
|
}
|
|
return effective
|
|
}
|
|
|
|
// GetModelAllowlist returns the model allowlist for a user.
|
|
func GetModelAllowlist(db *gorm.DB, userID string) ModelAllowlist {
|
|
perm, err := GetUserPermissions(db, userID)
|
|
if err != nil {
|
|
return ModelAllowlist{}
|
|
}
|
|
return perm.AllowedModels
|
|
}
|
|
|
|
// UpdateModelAllowlist updates the model allowlist for a user.
|
|
func UpdateModelAllowlist(db *gorm.DB, userID string, allowlist ModelAllowlist) error {
|
|
perm, err := GetUserPermissions(db, userID)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
perm.AllowedModels = allowlist
|
|
return db.Save(perm).Error
|
|
}
|
|
|
|
// IsModelAllowed returns true if the user is allowed to use the given model.
|
|
// Admins always have access. If the allowlist is not enabled, all models are allowed.
|
|
func IsModelAllowed(db *gorm.DB, user *User, modelName string) bool {
|
|
if user == nil {
|
|
return false
|
|
}
|
|
if user.Role == RoleAdmin {
|
|
return true
|
|
}
|
|
allowlist := GetModelAllowlist(db, user.ID)
|
|
if !allowlist.Enabled {
|
|
return true
|
|
}
|
|
for _, m := range allowlist.Models {
|
|
if m == modelName {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|