mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-14 11:49:33 -04:00
7a4ca8f60d
Adds a Go native gRPC backend that dlopens librfdetrcpp.so (built from
mudler/rf-detr.cpp at the pinned RFDETR_VERSION) via purego and exposes
the rfdetr.cpp inference pipeline through LocalAI's existing Detect RPC.
Supports all 5 RF-DETR detection variants (Nano/Small/Base/Medium/Large)
and 6 segmentation variants (SegNano/SegSmall/SegMedium/SegLarge/
SegXLarge/Seg2XLarge) with F32/F16/Q8_0/Q4_K quantizations. Pre-built
GGUFs ship at mudler/rfdetr-cpp-* on HuggingFace.
Detection returns Bbox + class_name + confidence; segmentation also
returns PNG-encoded per-detection masks via the rfdetr_capi accessor
functions (rfdetr_capi_get_detection_{class_id,box,score,class_name,
mask_png}).
End-to-end verified through POST /v1/detection: HTTP -> gRPC -> purego
dlopen -> rfdetr.cpp -> ggml -> response (9 detections on the detection
model, 21 detections + valid PNG masks on the seg-nano model against
the kitchen fixture).
Wiring:
- backend/go/rfdetr-cpp/{main.go,gorfdetrcpp.go,CMakeLists.txt,
Makefile,run.sh,package.sh,test.sh,.gitignore}
- Top-level Makefile: BACKEND_RFDETR_CPP, docker-build target,
.NOTPARALLEL, prepare-test-extra, test-extra
- backend/go/rfdetr-cpp/Makefile: `test` target invoked by test-extra
- .github/backend-matrix.yml: CPU + CUDA-12/13 + L4T CUDA-12/13
(arm64) + HIP + Vulkan (amd64 + arm64) + SYCL f32/f16
- backend/index.yaml: rfdetr-cpp meta anchor + latest/development
image entries for every matrix tag-suffix
- .github/workflows/bump_deps.yaml: RFDETR_VERSION pin tracking
(mudler/rf-detr.cpp branch main)
- gallery/index.yaml: 11 rfdetr-cpp-* entries (nano + 4 detection
variants + 6 seg variants), all backed by mudler/rfdetr-cpp-*
on HuggingFace with sha256 pinning on the F16 default
- core/gallery/importers/rfdetr.go: GGUF auto-routing for HF imports
(mudler/rfdetr-cpp-* repos route to rfdetr-cpp, Transformer-format
repos stay on the Python rfdetr backend; explicit preferences.backend
overrides both heuristics)
- core/gallery/importers/rfdetr_test.go: table-driven coverage of the
auto-routing + a live mudler/rfdetr-cpp-nano cross-check
scripts/changed-backends.js needs no change: the existing
Dockerfile.golang -> backend/go/${item.backend}/ branch already routes
the 9 rfdetr-cpp matrix entries to the correct backend path.
Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
80 lines
3.4 KiB
CMake
80 lines
3.4 KiB
CMake
cmake_minimum_required(VERSION 3.18)
|
|
project(librfdetrcpp LANGUAGES C CXX)
|
|
|
|
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
|
set(CMAKE_CXX_STANDARD 17)
|
|
set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
|
|
|
# Static-link ggml + rfdetr so the resulting .so has no runtime dependency on
|
|
# extra ggml/rfdetr shared libraries — only on libc/libstdc++/libgomp, which
|
|
# the LocalAI package step bundles into the docker image.
|
|
set(BUILD_SHARED_LIBS OFF CACHE BOOL "Build static libraries" FORCE)
|
|
|
|
# rfdetr.cpp build switches: skip CLI/tests, keep static lib.
|
|
set(RFDETR_BUILD_CLI OFF CACHE BOOL "Disable rfdetr CLI" FORCE)
|
|
set(RFDETR_BUILD_TESTS OFF CACHE BOOL "Disable rfdetr tests" FORCE)
|
|
set(RFDETR_SHARED OFF CACHE BOOL "Build rfdetr as static lib" FORCE)
|
|
|
|
# rt-detr.cpp's top-level CMakeLists invokes
|
|
# `bash ${CMAKE_SOURCE_DIR}/scripts/apply_ggml_patches.sh` to apply its
|
|
# in-tree ggml patches before descending into the submodule. When we
|
|
# `add_subdirectory` it from a parent project, `CMAKE_SOURCE_DIR` points
|
|
# at *our* directory, not theirs, so the script path resolves wrong.
|
|
#
|
|
# Run the patches script ourselves up front (it's idempotent — re-running
|
|
# is a no-op once patches are applied) so the rt-detr.cpp configure step
|
|
# is essentially a no-op for the patch hook.
|
|
set(RFDETR_CPP_SRC ${CMAKE_CURRENT_SOURCE_DIR}/sources/rt-detr.cpp)
|
|
if(EXISTS ${RFDETR_CPP_SRC}/scripts/apply_ggml_patches.sh)
|
|
execute_process(
|
|
COMMAND bash ${RFDETR_CPP_SRC}/scripts/apply_ggml_patches.sh
|
|
RESULT_VARIABLE _rfdetr_patch_result
|
|
OUTPUT_VARIABLE _rfdetr_patch_output
|
|
ERROR_VARIABLE _rfdetr_patch_error
|
|
OUTPUT_STRIP_TRAILING_WHITESPACE
|
|
ERROR_STRIP_TRAILING_WHITESPACE)
|
|
if(NOT _rfdetr_patch_result EQUAL 0)
|
|
message(FATAL_ERROR
|
|
"Failed to apply ggml patches (exit ${_rfdetr_patch_result}):\n"
|
|
"stdout:\n${_rfdetr_patch_output}\n"
|
|
"stderr:\n${_rfdetr_patch_error}")
|
|
endif()
|
|
message(STATUS "${_rfdetr_patch_output}")
|
|
endif()
|
|
|
|
# Stage a shim 'scripts/apply_ggml_patches.sh' under our source dir so that
|
|
# rt-detr.cpp's CMakeLists — which calls
|
|
# bash ${CMAKE_SOURCE_DIR}/scripts/apply_ggml_patches.sh
|
|
# — finds an idempotent no-op there. The real patches have already been
|
|
# applied above; this just satisfies the path lookup.
|
|
file(MAKE_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/scripts)
|
|
file(WRITE ${CMAKE_CURRENT_SOURCE_DIR}/scripts/apply_ggml_patches.sh
|
|
"#!/usr/bin/env bash
|
|
# Shim - patches were already applied by the parent CMakeLists.
|
|
exit 0
|
|
")
|
|
execute_process(COMMAND chmod +x ${CMAKE_CURRENT_SOURCE_DIR}/scripts/apply_ggml_patches.sh)
|
|
|
|
add_subdirectory(./sources/rt-detr.cpp)
|
|
|
|
# rfdetr.cpp's C-API symbols already live inside librfdetr (src/rfdetr_capi.cpp
|
|
# is compiled into the lib). We re-export them via a MODULE library that
|
|
# whole-archive-links rfdetr so the symbols are visible at dlopen time.
|
|
add_library(rfdetrcpp MODULE
|
|
sources/rt-detr.cpp/src/rfdetr_capi.cpp)
|
|
|
|
target_include_directories(rfdetrcpp PRIVATE
|
|
sources/rt-detr.cpp/include
|
|
sources/rt-detr.cpp/src
|
|
sources/rt-detr.cpp/third_party/stb
|
|
)
|
|
|
|
target_link_libraries(rfdetrcpp PRIVATE rfdetr ggml)
|
|
|
|
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9.0)
|
|
target_link_libraries(rfdetrcpp PRIVATE stdc++fs)
|
|
endif()
|
|
|
|
set_property(TARGET rfdetrcpp PROPERTY CXX_STANDARD 17)
|
|
set_target_properties(rfdetrcpp PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
|