mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-19 06:09:07 -04:00
* feat(backend): add depth-anything (Depth Anything 3) C++/ggml backend + gallery Mirrors the locate-anything-cpp backend to register a new depth-anything backend that wraps the Depth Anything 3 ggml port (depth-anything.cpp) via purego (cgo-less, no Python at inference). - backend/go/depth-anything-cpp/: gRPC backend (Load + Predict + GenerateImage), purego binding to the da_capi_* C ABI, CMake/Makefile/run/package/test scripts building depth-anything.cpp's DA_SHARED static .so per CPU variant. - backend/index.yaml: depth-anything backend meta + all hardware-variant capability entries (cpu/cuda12/cuda13/intel-sycl-f32+f16/vulkan/nvidia-l4t). - gallery/index.yaml: 8 Depth Anything 3 GGUF models (base q4_k/q8_0/f16/f32, small, large, giant, mono-large). - .github/backend-matrix.yml: one build entry per hardware variant. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(depth): typed Depth RPC + REST endpoint exposing full DA3 data Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(depth): pin depth-anything.cpp to e0b6814 (ABI 3 dense C-API) The Depth RPC handler calls da_capi_depth_dense / da_capi_points (C-API ABI 3); pin the native build to the commit that exports them. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(depth): pin depth-anything.cpp to v0.1.0 release (b515c31) Repoint the native version from the now-orphaned e0b6814 to the b515c31 release commit, kept alive by the upstream v0.1.0 tag. C-API is unchanged (da_capi_abi_version == 3). Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(depth): wire depth-anything-cpp into build, CI bump, and importer The backend dir, gallery index, and CI build-matrix were present but the backend was never wired into the integration points that adding-backends.md requires: - root Makefile: add to .NOTPARALLEL, the test-extra chain, a BACKEND_* definition, the docker-build target eval, and docker-build-backends (mirrors parakeet-cpp; the backend's own Makefile already documented that its `test` target is driven by test-extra). - bump_deps.yaml: register the DEPTHANYTHING_VERSION pin so the daily auto-bump bot tracks mudler/depth-anything.cpp master (it cannot see an unregistered Makefile pin). - import form: add a preference-only KnownBackend entry so depth-anything is selectable at /import-model (mirrors sam3-cpp; no reliable GGUF auto-detect signal, so pref-only per the doc's default). changed-backends.js needs no entry: the generic golang suffix branch already resolves backend/go/depth-anything-cpp/. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(depth): auto-detect importer for depth-anything GGUFs Replace the preference-only entry with a real auto-detect importer (mirrors parakeet-cpp / locate-anything): - DepthAnythingImporter matches a .gguf whose name carries a depth-anything token (depth-anything-<size>-<quant>.gguf), so /import-model recognises mudler/depth-anything.cpp-gguf repos and direct GGUF URLs without an explicit backend preference. preferences.backend= "depth-anything" still forces it. - Registered before LlamaCPPImporter so its GGUF bundles aren't claimed by the generic .gguf importer; the narrow name match means it cannot claim arbitrary llama GGUFs or the upstream safetensors PyTorch repos. - Multi-quant repos pick the smallest quant by default (q4_k -> ... -> f32, depth stays >0.998 corr even at q4_k); quantizations preference overrides. - Drops the now-redundant knownPrefOnlyBackends entry (importer-backed backends are not listed there, matching parakeet-cpp). - Table-driven Ginkgo test covers detection, negative cases (llama GGUF, upstream safetensors), default/override/fallback quant pick, and direct URL import. 10/10 specs pass. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(depth): check conn.Close error in grpc Depth client (errcheck) The new Depth() client method used a bare `defer conn.Close()`. golangci-lint runs with new-from-merge-base, so although the 39 sibling methods use the same bare form (grandfathered), the newly added line trips errcheck. Drop the result explicitly to satisfy the linter. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 * fix(depth): bump depth-anything.cpp to v0.1.1 (embeddable CMake) v0.1.0 (b515c31) used ${CMAKE_SOURCE_DIR} for its include dirs, which points at the parent project when built via add_subdirectory() as this backend does, so the container build failed with missing stb_image.h / da_gguf_keys.h. v0.1.1 (2d42897) switches to project-relative paths. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 * fix(depth): resolve gosec findings in the backend wrapper The code-scanning gate flagged three new failure-level alerts in godepthanythingcpp.go (gosec runs with -no-fail; GitHub gates on new alerts): - G301: export dirs were created with 0o755. Tighten to 0o750 (no world access needed for backend-written export output). - G304: writeDepthPNG creates req.GetDst(). That path is chosen by the LocalAI core as the intended output destination (same pattern every image backend uses), not attacker input, so annotate with #nosec G304 and document why. The remaining G103 "audit unsafe" notes on the unsafe.Slice C-buffer copies are warning-level (the same purego interop whisper/parakeet use) and do not gate the check, per the supertonic exclusion precedent in secscan.yaml. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 * fix(depth): bump depth-anything.cpp to v0.1.2 (CUDA cross-build arch) v0.1.1 forced CMAKE_CUDA_ARCHITECTURES=native, which breaks the GPU-less l4t/cublas CI builds (nvcc "Unsupported gpu architecture 'compute_'" on CMake 3.22). v0.1.2 (442eea4) drops the override and lets ggml pick its default cross-build arch list. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
168 lines
5.7 KiB
Go
168 lines
5.7 KiB
Go
package main
|
|
|
|
// main_test.go - end-to-end smoke test for the depth-anything-cpp gRPC backend.
|
|
//
|
|
// Spawns the compiled depth-anything-cpp binary on a free local port, dials it
|
|
// via gRPC, and exercises LoadModel + Predict against the test fixtures
|
|
// downloaded by test.sh: the small (vits) f32 GGUF of Depth Anything 3 and a
|
|
// real photo. Asserts that Predict returns a JSON payload with a positive
|
|
// depth-map width/height.
|
|
//
|
|
// The spec Skip()s cleanly if its fixtures (the model, the test image, the
|
|
// built binary, or the fallback .so) are missing, so the test target stays
|
|
// usable on a fresh checkout / on CI runners where the model hasn't been
|
|
// downloaded.
|
|
|
|
import (
|
|
"context"
|
|
"encoding/base64"
|
|
"encoding/json"
|
|
"fmt"
|
|
"net"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"testing"
|
|
"time"
|
|
|
|
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
|
|
. "github.com/onsi/ginkgo/v2"
|
|
. "github.com/onsi/gomega"
|
|
"google.golang.org/grpc"
|
|
"google.golang.org/grpc/credentials/insecure"
|
|
)
|
|
|
|
func TestDepth(t *testing.T) {
|
|
RegisterFailHandler(Fail)
|
|
RunSpecs(t, "depth-anything-cpp backend smoke suite")
|
|
}
|
|
|
|
// freePort grabs an ephemeral TCP port and immediately releases it so the
|
|
// spawned backend can bind to it. There is a tiny TOCTOU window here but in
|
|
// practice it's adequate for a smoke test on a quiet runner.
|
|
func freePort() int {
|
|
l, err := net.Listen("tcp", "127.0.0.1:0")
|
|
Expect(err).ToNot(HaveOccurred(), "freePort listen")
|
|
port := l.Addr().(*net.TCPAddr).Port
|
|
Expect(l.Close()).To(Succeed())
|
|
return port
|
|
}
|
|
|
|
// startBackend spawns the depth-anything-cpp binary on the given port and waits
|
|
// until it accepts TCP connections (up to 10s). It mirrors how main.go resolves
|
|
// the purego library: the DEPTHANYTHING_LIBRARY env var points the dlopen at the
|
|
// freshly built fallback .so. The returned cleanup func kills the process.
|
|
func startBackend(port int) func() {
|
|
binary, err := filepath.Abs("./depth-anything-cpp")
|
|
Expect(err).ToNot(HaveOccurred())
|
|
if _, err := os.Stat(binary); err != nil {
|
|
Skip(fmt.Sprintf("backend binary not built: %s (run `make depth-anything-cpp` first)", binary))
|
|
}
|
|
|
|
libPath, err := filepath.Abs("./libdepthanythingcpp-fallback.so")
|
|
Expect(err).ToNot(HaveOccurred())
|
|
if _, err := os.Stat(libPath); err != nil {
|
|
Skip(fmt.Sprintf("fallback library not built: %s (run `make libdepthanythingcpp-fallback.so` first)", libPath))
|
|
}
|
|
|
|
addr := fmt.Sprintf("127.0.0.1:%d", port)
|
|
cmd := exec.Command(binary, "--addr", addr)
|
|
cmd.Env = append(os.Environ(), "DEPTHANYTHING_LIBRARY="+libPath)
|
|
cmd.Stdout = os.Stderr
|
|
cmd.Stderr = os.Stderr
|
|
Expect(cmd.Start()).To(Succeed())
|
|
|
|
cleanup := func() {
|
|
if cmd.Process != nil {
|
|
_ = cmd.Process.Kill()
|
|
_, _ = cmd.Process.Wait()
|
|
}
|
|
}
|
|
|
|
deadline := time.Now().Add(10 * time.Second)
|
|
for time.Now().Before(deadline) {
|
|
c, err := net.DialTimeout("tcp", addr, 200*time.Millisecond)
|
|
if err == nil {
|
|
_ = c.Close()
|
|
return cleanup
|
|
}
|
|
time.Sleep(200 * time.Millisecond)
|
|
}
|
|
|
|
cleanup()
|
|
Fail(fmt.Sprintf("backend did not become ready on %s within 10s", addr))
|
|
return func() {}
|
|
}
|
|
|
|
// loadTestImage reads the test image downloaded by test.sh and returns its
|
|
// base64-encoded content (one of the wire formats accepted by Predict).
|
|
func loadTestImage() string {
|
|
imgPath, err := filepath.Abs("test-data/test.jpg")
|
|
Expect(err).ToNot(HaveOccurred())
|
|
imgBytes, err := os.ReadFile(imgPath)
|
|
if err != nil {
|
|
Skip(fmt.Sprintf("test image not present: %s (run test.sh first)", imgPath))
|
|
}
|
|
return base64.StdEncoding.EncodeToString(imgBytes)
|
|
}
|
|
|
|
// dialBackend opens a gRPC client connection to the spawned backend.
|
|
func dialBackend(port int) (pb.BackendClient, func()) {
|
|
addr := fmt.Sprintf("127.0.0.1:%d", port)
|
|
conn, err := grpc.NewClient(addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
|
Expect(err).ToNot(HaveOccurred())
|
|
return pb.NewBackendClient(conn), func() { _ = conn.Close() }
|
|
}
|
|
|
|
// modelPathOrSkip resolves the model file under ./test-models/ and Skip()s the
|
|
// current spec if it's missing (not present on a fresh checkout / on CI runners
|
|
// without the download).
|
|
func modelPathOrSkip(name string) string {
|
|
modelDir, err := filepath.Abs("test-models")
|
|
Expect(err).ToNot(HaveOccurred())
|
|
modelPath := filepath.Join(modelDir, name)
|
|
if _, err := os.Stat(modelPath); err != nil {
|
|
Skip(fmt.Sprintf("model not present: %s (run test.sh first)", modelPath))
|
|
}
|
|
return modelPath
|
|
}
|
|
|
|
var _ = Describe("depth-anything-cpp backend", func() {
|
|
It("runs depth+pose against a known-good image", func() {
|
|
modelPath := modelPathOrSkip("depth-anything-small-f32.gguf")
|
|
imgB64 := loadTestImage()
|
|
|
|
port := freePort()
|
|
cleanup := startBackend(port)
|
|
defer cleanup()
|
|
|
|
client, closeConn := dialBackend(port)
|
|
defer closeConn()
|
|
|
|
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
|
|
defer cancel()
|
|
|
|
loadResp, err := client.LoadModel(ctx, &pb.ModelOptions{
|
|
Model: "depth-anything-small-f32.gguf",
|
|
ModelFile: modelPath,
|
|
Threads: 4,
|
|
})
|
|
Expect(err).ToNot(HaveOccurred(), "LoadModel")
|
|
Expect(loadResp.GetSuccess()).To(BeTrue(), "LoadModel reported failure: %s", loadResp.GetMessage())
|
|
|
|
// Predict runs depth+pose and returns the JSON depthResult in Reply.Message.
|
|
reply, err := client.Predict(ctx, &pb.PredictOptions{
|
|
Images: []string{imgB64},
|
|
})
|
|
Expect(err).ToNot(HaveOccurred(), "Predict")
|
|
|
|
var res depthResult
|
|
Expect(json.Unmarshal(reply.GetMessage(), &res)).To(Succeed(), "Predict returned non-JSON: %q", string(reply.GetMessage()))
|
|
Expect(res.DepthW).To(BeNumerically(">", 0), "depth width should be positive")
|
|
Expect(res.DepthH).To(BeNumerically(">", 0), "depth height should be positive")
|
|
|
|
_, _ = fmt.Fprintf(GinkgoWriter, "depth OK: %dx%d min=%.3f max=%.3f\n",
|
|
res.DepthW, res.DepthH, res.DepthMin, res.DepthMax)
|
|
})
|
|
})
|