mirror of
https://github.com/mudler/LocalAI.git
synced 2026-05-30 11:36:31 -04:00
7a4ca8f60d
Adds a Go native gRPC backend that dlopens librfdetrcpp.so (built from
mudler/rf-detr.cpp at the pinned RFDETR_VERSION) via purego and exposes
the rfdetr.cpp inference pipeline through LocalAI's existing Detect RPC.
Supports all 5 RF-DETR detection variants (Nano/Small/Base/Medium/Large)
and 6 segmentation variants (SegNano/SegSmall/SegMedium/SegLarge/
SegXLarge/Seg2XLarge) with F32/F16/Q8_0/Q4_K quantizations. Pre-built
GGUFs ship at mudler/rfdetr-cpp-* on HuggingFace.
Detection returns Bbox + class_name + confidence; segmentation also
returns PNG-encoded per-detection masks via the rfdetr_capi accessor
functions (rfdetr_capi_get_detection_{class_id,box,score,class_name,
mask_png}).
End-to-end verified through POST /v1/detection: HTTP -> gRPC -> purego
dlopen -> rfdetr.cpp -> ggml -> response (9 detections on the detection
model, 21 detections + valid PNG masks on the seg-nano model against
the kitchen fixture).
Wiring:
- backend/go/rfdetr-cpp/{main.go,gorfdetrcpp.go,CMakeLists.txt,
Makefile,run.sh,package.sh,test.sh,.gitignore}
- Top-level Makefile: BACKEND_RFDETR_CPP, docker-build target,
.NOTPARALLEL, prepare-test-extra, test-extra
- backend/go/rfdetr-cpp/Makefile: `test` target invoked by test-extra
- .github/backend-matrix.yml: CPU + CUDA-12/13 + L4T CUDA-12/13
(arm64) + HIP + Vulkan (amd64 + arm64) + SYCL f32/f16
- backend/index.yaml: rfdetr-cpp meta anchor + latest/development
image entries for every matrix tag-suffix
- .github/workflows/bump_deps.yaml: RFDETR_VERSION pin tracking
(mudler/rf-detr.cpp branch main)
- gallery/index.yaml: 11 rfdetr-cpp-* entries (nano + 4 detection
variants + 6 seg variants), all backed by mudler/rfdetr-cpp-*
on HuggingFace with sha256 pinning on the F16 default
- core/gallery/importers/rfdetr.go: GGUF auto-routing for HF imports
(mudler/rfdetr-cpp-* repos route to rfdetr-cpp, Transformer-format
repos stay on the Python rfdetr backend; explicit preferences.backend
overrides both heuristics)
- core/gallery/importers/rfdetr_test.go: table-driven coverage of the
auto-routing + a live mudler/rfdetr-cpp-nano cross-check
scripts/changed-backends.js needs no change: the existing
Dockerfile.golang -> backend/go/${item.backend}/ branch already routes
the 9 rfdetr-cpp matrix entries to the correct backend path.
Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
56 lines
2.2 KiB
Bash
Executable File
56 lines
2.2 KiB
Bash
Executable File
#!/bin/bash
|
|
set -e
|
|
|
|
CURDIR=$(dirname "$(realpath $0)")
|
|
|
|
echo "Running rfdetr-cpp backend tests..."
|
|
|
|
# Test models from the mudler/rfdetr-cpp-* HuggingFace repos. Both the
|
|
# detection (nano-q8_0, ~36 MB) and segmentation (seg-nano-q8_0, ~40 MB)
|
|
# variants are downloaded so the Go smoke test exercises both code paths.
|
|
RFDETR_MODEL_DIR="${RFDETR_MODEL_DIR:-$CURDIR/test-models}"
|
|
|
|
RFDETR_DET_FILE="${RFDETR_DET_FILE:-rfdetr-nano-q8_0.gguf}"
|
|
RFDETR_DET_URL="${RFDETR_DET_URL:-https://huggingface.co/mudler/rfdetr-cpp-nano/resolve/main/rfdetr-nano-q8_0.gguf}"
|
|
|
|
RFDETR_SEG_FILE="${RFDETR_SEG_FILE:-rfdetr-seg-nano-q8_0.gguf}"
|
|
RFDETR_SEG_URL="${RFDETR_SEG_URL:-https://huggingface.co/mudler/rfdetr-cpp-seg-nano/resolve/main/rfdetr-seg-nano-q8_0.gguf}"
|
|
|
|
mkdir -p "$RFDETR_MODEL_DIR"
|
|
|
|
if [ ! -f "$RFDETR_MODEL_DIR/$RFDETR_DET_FILE" ]; then
|
|
echo "Downloading rfdetr nano-q8_0 detection model..."
|
|
curl -L -o "$RFDETR_MODEL_DIR/$RFDETR_DET_FILE" "$RFDETR_DET_URL" --progress-bar
|
|
fi
|
|
|
|
if [ ! -f "$RFDETR_MODEL_DIR/$RFDETR_SEG_FILE" ]; then
|
|
echo "Downloading rfdetr seg-nano-q8_0 segmentation model..."
|
|
curl -L -o "$RFDETR_MODEL_DIR/$RFDETR_SEG_FILE" "$RFDETR_SEG_URL" --progress-bar
|
|
fi
|
|
|
|
# Use a real COCO test image from the upstream rf-detr.cpp repo (~46 KB).
|
|
# A synthetic 64x64 red PNG was too synthetic to elicit detections from a
|
|
# real model — the smoke test would always trivially pass with zero
|
|
# detections.
|
|
TEST_IMAGE_DIR="$CURDIR/test-data"
|
|
TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg"
|
|
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
|
|
|
mkdir -p "$TEST_IMAGE_DIR"
|
|
if [ ! -f "$TEST_IMAGE_FILE" ]; then
|
|
echo "Downloading COCO test image..."
|
|
curl -L -o "$TEST_IMAGE_FILE" "$TEST_IMAGE_URL" --progress-bar
|
|
fi
|
|
|
|
echo "rfdetr-cpp test setup complete."
|
|
echo " detection model: $RFDETR_MODEL_DIR/$RFDETR_DET_FILE"
|
|
echo " segmentation model: $RFDETR_MODEL_DIR/$RFDETR_SEG_FILE"
|
|
echo " test image: $TEST_IMAGE_FILE"
|
|
|
|
# Run the Go smoke test: spawns the backend binary on a free port, calls
|
|
# LoadModel + Detect via gRPC for both detection and segmentation models.
|
|
echo ""
|
|
echo "Running Go smoke test..."
|
|
cd "$CURDIR"
|
|
go test -v -timeout 5m ./...
|