mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-05 04:24:39 -04:00
mudler-agentandEttore Di Giacinto
99043b442c
* fix(schema): preserve SystemOne image inputs Assisted-by: OpenAI * test(schema): follow Ginkgo conventions for decision inputs Assisted-by: OpenAI * feat(llama-cpp): dispatch native decisions through Score Upgrade the stock dependency and reconcile Score/TTS patches. Reuse native decision parsing, tasks, formatting and response-reader cleanup; preserve ordinary scoring admission and guard older dependencies. Assisted-by: OpenAI * refactor(systemone): share request and model validation Assisted-by: OpenAI:gpt-5 * fix(systemone): preserve HTTP wire-byte validation limit Keep structural validation separate from the serialized internal request bound so HTML escaping cannot reject valid HTTP payloads. Assisted-by: OpenAI:gpt-5 * feat(systemone): bound images and account native decisions Preserve public wire limits independently from router serialization. Reject unsupported NER images, map native request/capability errors, and stamp explicit usage once. Advertise decisions for stock llama-cpp. Assisted-by: OpenAI * fix(systemone): record usage on registered native route Exercise real registration and billing with a mock native backend. Reject empty native responses, malformed image URLs, trailing JSON, and wire overflow including whitespace. Assisted-by: OpenAI * feat(router): add lazy native decision transport Bind named models through internal ModelSystemOne calls with shared validation and bounded abandoned operations. Remove request and echoed-error contents from decision traces. Assisted-by: OpenAI:gpt-5 * feat(router): classify overlapping policies with native decisions Ask independent noul questions, validate probabilities and preserve first-superset routing. Wire the central factory with config-sensitive invalidation and cancellation-safe resolution. Document native framing and bounded operation limits. Assisted-by: OpenAI:gpt-5 * feat(gallery): add pinned Julia-1 native decision model Add a separate text-only llama-cpp Q8 entry with pinned Apache-2.0 source provenance and checksum. Installed using the gallery installer and exercised choice, score and noul on CPU. Assisted-by: OpenAI * test(router): verify native decisions through central factory Add an opt-in real-model Ginkgo integration covering the native Go loader and C++ transport, token usage, independent overlapping labels, and candidate selection. Document owned-server execution and the intentionally non-quality threshold. Assisted-by: Codex:gpt-5 * fix(llama-cpp): align upstream pin and preserve decision signatures Advance to bed0a856 without losing the automated upstream bump. Detect full-request fill_task support at compile time and forward every question for Nimble framing while retaining the earlier native signature. Preserve reconciled SCORE/TTS patches; add standalone compatibility coverage. Assisted-by: Codex:gpt-5 * feat(gallery): add native decision family defaults Pin Laya, Kev-4B, lev, OpenJev and Nimble artifacts. Verify Laya/Kev/lev gallery installs and CPU contracts on both native pins; clearly mark OpenJev/Nimble runtime validation pending and their noncommercial licenses. Assisted-by: OpenAI * docs(decisions): clarify integrated Nimble prerequisite Record the exact combined backend pin while retaining pending OpenJev and Nimble installation/runtime validation status. Assisted-by: Codex:gpt-5 * fix(gallery): indent native decision model sequences Match repository yamllint indentation for Laya, Kev, lev and OpenJev list fields. Parsed gallery data is unchanged; reproduce CI gallery lint failure before the whitespace-only fix and pass the same command afterward. Assisted-by: Codex:gpt-5 * docs(decisions): record OpenJev and Nimble CPU validation Record gallery installation, checksum/metadata verification and multiquestion native smoke results on bed0a856. Retain noncommercial and text-only limitations without accuracy or deterministic-output claims. Assisted-by: OpenAI * fix(ui): expose native Decisions router classifiers Select classifier models using metadata-driven capability routing, retain tuned thresholds, and validate native decision selections before saving. Cover both native backends and create/save/reopen in the real React editor. Assisted-by: Codex:gpt-5 * fix(router): exclude aliases from native decision discovery Check the originally named config before advertising native Decisions eligibility. Retain target capability inheritance for ordinary generation aliases. Exercise the actual capabilities endpoint with native models on both backends, aliases, and disabled models. Assisted-by: Codex:gpt-5 * feat(systemone): share bounded multimodal input validation Preserve text wire limits while admitting bounded PNG/JPEG decision input. Share collection and header validation across internal and public callers and keep the native runner response budget independent. Assisted-by: OpenAI:API-assistant * fix(systemone): bound admission lifetimes and validate complete images Retain shared admission leases through actual work completion, including abandoned internal operations. Decode bounded image pixels, cap public native responses before usage stamping, and preserve oversized malformed text status precedence. Assisted-by: OpenAI:API-assistant * fix(router): classify images before media fetching Preserve ordered structured probes for native decisions. Defer OpenAI media preparation until routing selects the served model, so rejected decision URLs cannot trigger downloads before shared validation. Guard direct image collection with context-aware shared admission. Keep text classifiers and embedding caches from discarding image input. Retain fail-closed classifier configuration and runtime fallback policy. Add middleware, typed-content, admission, cancellation and cache tests. Assisted-by: OpenAI:API-assistant * fix(router): bound extraction before serialization Check probe budgets before copying text or marshaling message state. Count JSON escaping so oversized internal inputs fail before allocation. Preserve typed Anthropic blocks through selected-model conversion and fallback. Keep retry coverage in Ginkgo without global test registration. Assisted-by: OpenAI * fix(router): bound supported probe serialization Arbitrary structs can bypass the probe budget through pointer marshalers, string tags, and promoted fields. Accept concrete chat schema types and plain JSON values instead of emulating arbitrary struct serialization. Budget escaped direct prompts before marshaling so raw length cannot hide serialized expansion. Preserve runtime fallback and reject oversized input before invoking the decision runner. Add Ginkgo allocation, boundary, and marshaler invocation regressions. Six-package tests, three-package race tests, and full-T2 delta lint pass. Assisted-by: OpenAI:GPT-5 golangci-lint * feat(decisions): enable bounded OpenJev images Validate native decision images before permissive media parsing and pixel allocation. Require both decision image support and a vision projector; missing or audio-only projectors cannot silently become text decisions. Pin the OpenJev Q8 projector and document its license and disk footprint. Add native safety tests, canonical limit parity, gallery and load-option checks, and a reproducible CPU direct-RPC contrasting-image smoke. Assisted-by: OpenAI:GPT-5 * fix(decisions): reject incomplete image streams stb accepts corrupt PNG Adler checksums and truncated JPEG scans. Use bounded zlib validation and strict libjpeg decoding before parsing. Keep dimension and aggregate pixel checks ahead of decoder allocations. Wire decoder dependencies into native builds and runtime packaging. Add regressions for appended EOI and embedded marker bypasses. Assisted-by: OpenAI:GPT-5 * fix(ci): gate native decision image validation Run the decoder security tests outside the stdlib-only native suite. Fetch vendor headers at the backend pin and provision decoder dependencies. Gate Go limit parity and production CMake wiring without model downloads. Assisted-by: OpenAI:GPT-5 * test(decisions): cover multimodal public API paths Exercise shared image contracts through the registered HTTP routes and external mock backend. Add opt-in cached gallery installation and real OpenJev image decisions through SystemOne and both routing APIs. Assisted-by: Codex:gpt-5 * test(decisions): assert isolation and cache bypass Observe external RPC calls and compare complete classifier history. Winner-only and cache-miss checks could hide dropped history or cache use. Give real inference its own application and model directory so shared backend mappings and loaded processes cannot affect mixed suite order. Assisted-by: OpenAI:ChatGPT * test(decisions): isolate fixture globals Disable optional global services in the isolated HTTP fixture and register cleanup before setup assertions. Verify meter provider identity survives fixture creation and destruction. Snapshot observed usage before assertions so failures cannot retain the mutex. Require a successful usage stamp before checking error responses. Assisted-by: Codex:gpt-5 golangci-lint * fix(application): honor optional telemetry controls Skip failover gauge registration when metrics are disabled. Register against the application meter rather than looking up the global provider. Allow embedders to retain the bounded routing log without billing stats. Keep the existing default when stats are disabled. The isolated HTTP fixture uses this option without losing its native router assertions. Assisted-by: Codex:gpt-5 golangci-lint --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
97 lines
4.8 KiB
Markdown
97 lines
4.8 KiB
Markdown
# Native decision bridge validation
|
|
|
|
The stock dependency is pinned to `bed0a856606ee4a24a164066f73d2379447033f5`.
|
|
`Score(question_type="systemone")` uses upstream decision tasks internally, not
|
|
HTTP. Plain Score keeps its existing admission checks. Older dependencies without
|
|
`server-decision.cpp` return gRPC `UNIMPLEMENTED` for this request type.
|
|
|
|
## Decision signature compatibility
|
|
|
|
This pin includes upstream Nimble support, in addition to OpenJev, Lev, Kev,
|
|
and Laya. The native bridge forwards the complete parsed question collection
|
|
when upstream's `fill_task` accepts it, as required by Nimble's schema framing.
|
|
`decision_compat.h` detects the callable C++ signature at compile time; older
|
|
native-decision forks still use their original single-question signature.
|
|
Forks without native decision support retain the existing `UNIMPLEMENTED` guard.
|
|
The standalone `decision_compat_test.cpp` checks both signatures and that the
|
|
full collection is passed by reference, not replaced with a singleton.
|
|
It is automatically discovered by `backend/cpp/run-unit-tests.sh`.
|
|
|
|
Signature and compile validation do not establish Nimble model accuracy or
|
|
runtime support for every artifact. Nimble weights are not part of this test
|
|
fixture. The official `ggml-org/Bespoke-Nimble-9B-v3-GGUF` model card declares
|
|
CC-BY-NC-4.0; check its restrictions before deployment.
|
|
|
|
## CPU build
|
|
|
|
Use a fresh stock checkout at the pin in `backend/cpp/llama-cpp/llama.cpp`.
|
|
Do not reuse a customized developer checkout. Apply patches once:
|
|
|
|
```sh
|
|
cd backend/cpp/llama-cpp/llama.cpp
|
|
git apply --check ../patches/0001-add-server-task-type-score.patch
|
|
git apply ../patches/0001-add-server-task-type-score.patch
|
|
git apply --check ../patches/0002-add-server-task-type-tts.patch
|
|
git apply ../patches/0002-add-server-task-type-tts.patch
|
|
cmake -S . -B build-cpu -DGGML_NATIVE=OFF -DLLAMA_OPENSSL=OFF \
|
|
-DLLAMA_CURL=OFF -DBUILD_SHARED_LIBS=OFF
|
|
cmake --build build-cpu --target llama-server -j2
|
|
```
|
|
|
|
For the normal product build, start instead from a fresh unpatched checkout and
|
|
run `make -C backend/cpp/llama-cpp grpc-server JOBS=2`; preparation applies the
|
|
patches and stages the bridge. This requires CMake packages for gRPC, protobuf,
|
|
and Abseil, plus `protoc` and `grpc_cpp_plugin`.
|
|
|
|
`build-decision-bridge.sh` is an alternative link validation for distro packages
|
|
that lack `ProtobufConfig.cmake`. It uses the already patched CPU static libraries
|
|
and the bridge source staged by `prepare.sh`. Do not apply patches twice when
|
|
staging that source. Set `CPU_BUILD` to the absolute `build-cpu` directory,
|
|
`OUT_DIR` to a scratch output directory, and optionally `DEPS_ROOT` to the root
|
|
of **locally extracted** distro packages. It does not install or download anything.
|
|
It also generates Python bindings, requiring `grpc_python_plugin`.
|
|
|
|
## Direct RPC smoke
|
|
|
|
The upstream test fixture `ggml-org/tinylaya-for-testing-gguf` has file
|
|
`tinylaya-for-testing-Q8_0.gguf`, size **97,200,288 bytes**, SHA-256
|
|
`a8b2b8f7fe6b7e10a884c55bf72362b0a8701e40dc3f332831d58246e5fa0b70`.
|
|
This is a test model, not a production gallery recommendation.
|
|
|
|
Start the backend with cores disabled, using the matching library path when
|
|
validating extracted distro dependencies:
|
|
|
|
```sh
|
|
ulimit -c 0
|
|
export LD_LIBRARY_PATH="$DEPS_ROOT/usr/lib/x86_64-linux-gnu${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
|
"$OUT_DIR/grpc-server" --addr 127.0.0.1:50051
|
|
```
|
|
|
|
In another terminal (Python requires `grpcio` and `protobuf`):
|
|
|
|
```sh
|
|
PYTHONPATH="$OUT_DIR" python3 backend/cpp/llama-cpp/tests/decision_smoke.py \
|
|
--address 127.0.0.1:50051 --model "$MODEL_FILE"
|
|
```
|
|
|
|
The smoke covers multiquestion choice/score/noul, normalized probabilities,
|
|
positive input and explicit zero output usage, concurrent calls, invalid JSON,
|
|
client cancellation/recovery, ordinary Score disabled/enabled, and missing
|
|
metadata. It reloads models; do not share the backend with another test runner.
|
|
The missing-metadata test makes a temporary equal-length metadata-key rename of
|
|
the fixture and explicitly enables embeddings, as appropriate for this encoder.
|
|
|
|
Limits: immediate client cancellation does not prove interruption during active
|
|
evaluation. The tiny fixture finishes too quickly for a deterministic timing-only
|
|
assertion; a queue barrier or server-side instrumentation is needed for that gate.
|
|
TTS is compiled and linked, not runtime-tested by this text-only fixture. Older
|
|
pin compile validation does not establish every supported fork's full build.
|
|
For bounded image/projector support and its separate runtime checks, see
|
|
[README-decision-images.md](README-decision-images.md).
|
|
|
|
The metadata-stripped encoder with embeddings disabled and `-np 1` aborts
|
|
in warmup at `llama-context.cpp`'s output-budget assertion on **clean unpatched**
|
|
upstream at the pinned revision as well. This is a preexisting invalid-fixture
|
|
configuration hazard, not a decision-dispatch or Score/TTS patch regression.
|
|
Do not use that configuration as the missing-metadata test.
|