mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 09:57:14 -04:00
f0d0bff232
The llama-cpp gRPC backend reconstructs OpenAI messages from proto for the tokenizer-template path and blindly json::parse'd each message's content string. LocalAI's Go layer always flattens content to a plain string, so a user prompt that merely looks like JSON (e.g. mealie's ingredient array ["1/4 cup brown sugar", ...]) was reinterpreted as structured content parts and rejected by oaicompat_chat_params_parse with "unsupported content[].type". Normalize content per role instead: user/system/developer content is opaque text and is never JSON-sniffed; assistant/tool content still collapses a literal JSON null/object (tool-call bookkeeping) to a string, but a plain string is never turned into an array/scalar. The array defense is role-independent, so the role gate only governs the benign null/object case. While here, extract the duplicated per-message reconstruction and the pre-template content sanitization into shared, unit-tested helpers (message_content.h) so the streaming (PredictStream) and non-streaming (Predict) paths cannot drift. This removes ~490 lines of copy-pasted defensive code, the dead tool-role parse branches, and the redundant Predict-only tool_calls branch, while preserving the prior #7324 (null content -> "") and #7528 (tool array content -> string) fixes. Tests: - backend/cpp/llama-cpp/message_content_test.cpp: standalone C++ unit tests for all three helpers (#10524, #7324, #7528, multimodal), discovered and run by `make test-backend-cpp` and a new generic tests-backend-cpp CI job. Also wired as an opt-in CMake/ctest target (-DLLAMA_GRPC_BUILD_TESTS=ON). - core/schema/message_test.go: Go regression pinning that ToProto flattens a JSON-array-looking text part to the verbatim string. - prepare.sh now copies message_content.h into the build tree. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
72 lines
2.4 KiB
Bash
Executable File
72 lines
2.4 KiB
Bash
Executable File
#!/bin/bash
|
|
#
|
|
# Discovers and runs every standalone C++ unit test under backend/cpp/.
|
|
#
|
|
# A "standalone" unit test is a *_test.cpp that depends only on the C++ standard
|
|
# library and nlohmann/json (single header) - i.e. it exercises pure helpers and
|
|
# does not need the full llama.cpp + gRPC backend build. Tests that DO need the
|
|
# backend build use the CMake/ctest path (e.g. -DLLAMA_GRPC_BUILD_TESTS=ON)
|
|
# instead and are skipped here.
|
|
#
|
|
# This keeps CI generic: adding a new pure-C++ unit test file named *_test.cpp in
|
|
# an active backend source dir is picked up automatically, with no CI edits.
|
|
#
|
|
# Env:
|
|
# NLOHMANN_INCLUDE include dir that contains nlohmann/json.hpp. If unset, the
|
|
# nlohmann/json single header is fetched to a temp dir.
|
|
# CXX compiler (default: g++).
|
|
# JSON_VERSION nlohmann/json tag to fetch when NLOHMANN_INCLUDE is unset
|
|
# (default: v3.11.3).
|
|
set -uo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "$0")" && pwd)"
|
|
CXX="${CXX:-g++}"
|
|
JSON_VERSION="${JSON_VERSION:-v3.11.3}"
|
|
|
|
JSON_INC="${NLOHMANN_INCLUDE:-}"
|
|
if [ -z "$JSON_INC" ]; then
|
|
JSON_INC="$(mktemp -d)"
|
|
mkdir -p "$JSON_INC/nlohmann"
|
|
echo "Fetching nlohmann/json ${JSON_VERSION} single header..."
|
|
if ! curl -L -sf \
|
|
"https://raw.githubusercontent.com/nlohmann/json/${JSON_VERSION}/single_include/nlohmann/json.hpp" \
|
|
-o "$JSON_INC/nlohmann/json.hpp"; then
|
|
echo "ERROR: failed to fetch nlohmann/json header" >&2
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
# Active source dirs only - exclude per-variant build copies, dev snapshots and
|
|
# the vendored upstream llama.cpp tree.
|
|
mapfile -t tests < <(find "$ROOT" -name '*_test.cpp' \
|
|
-not -path '*/llama.cpp/*' \
|
|
-not -path '*-build/*' \
|
|
-not -path '*-dev/*' \
|
|
-not -path '*fallback*' | sort)
|
|
|
|
if [ "${#tests[@]}" -eq 0 ]; then
|
|
echo "No standalone C++ unit tests found under $ROOT"
|
|
exit 0
|
|
fi
|
|
|
|
fail=0
|
|
for test_src in "${tests[@]}"; do
|
|
name="$(basename "$test_src" .cpp)"
|
|
bin="$(mktemp -d)/$name"
|
|
echo "==> $test_src"
|
|
if ! "$CXX" -std=c++17 -Wall -Wextra \
|
|
-I"$JSON_INC" -I"$(dirname "$test_src")" \
|
|
"$test_src" -o "$bin"; then
|
|
echo "COMPILE FAILED: $test_src" >&2
|
|
fail=1
|
|
continue
|
|
fi
|
|
if ! "$bin"; then
|
|
echo "TEST FAILED: $test_src" >&2
|
|
fail=1
|
|
fi
|
|
done
|
|
|
|
echo "Ran ${#tests[@]} standalone C++ unit test file(s)"
|
|
exit "$fail"
|