mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-19 06:09:07 -04:00
feat(pii): NER tier engine — privacy-filter.cpp backend + NER-centric PII filter (#10360)
Squashed feat/pii-ner-tier-engine rebased onto master (was 45 commits; see backup/pii-ner-tier-engine-prerebase). Net change: - privacy-filter.cpp: standalone GGML engine for the openai-privacy-filter PII/NER token classifier, wired as a LocalAI gRPC backend (CPU/CUDA/Vulkan). TokenClassify moves off the patched llama.cpp path onto this backend. - PII filter reworked to be NER-centric (encoder/NER detection tier scanning whole conversations as one document), with a recreated bounded restricted- regex secret-matching pattern detector tier alongside it (per-model pii_detection.builtins / .patterns + core/services/routing/piipattern). - Detection labelled by source (ner vs pattern); backend trace / confidence / debug observability; analyze/redact exposed as a synchronous API. - Instance-wide default detector policy + per-usecase default-on; request filtering extended to completions, embeddings, edits & Ollama. - React UI: NER-centric PII editor, detector-models table, pattern/builtins editor, middleware default-policy UI. - Gallery: privacy-filter-multilingual token-classify model + NER install filter; token_classify known_usecase; batch sized to context for NER models. privacy-filter backend registered in the backend gallery (cpu/vulkan/cuda-13 meta + image entries with a capabilities map) matching its CI matrix jobs, and an /import-model auto-detect importer (PrivacyFilterImporter, narrow privacy-filter GGUF detection) replacing the prior pref-only registration. Reconciled against master's independent evolution: - Dropped master's PIIPatternOverrides feature (global-pattern runtime overrides + /api/pii/patterns API + runtime_settings.json persistence). The per-model NER + pattern-detector design supersedes it; it was built on the global redactor pattern set this branch replaced. - Reverted the llama.cpp Score carry-patch (0006-server-task-type-score): removed the patch and restored master's grpc-server.cpp Score RPC (direct llama_decode, slot-loop bypass) and LLAMA_VERSION pin, plus master's model_config validation forbidding score + chat/completion/embeddings on llama-cpp. token_classify is unaffected (it runs on the privacy-filter backend, not llama-cpp). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com>
This commit is contained in:
committed by
GitHub
parent
c133ca39dc
commit
3fa7b2955c
@@ -34,6 +34,8 @@ const (
|
||||
BackendTraceAudioTransform BackendTraceType = "audio_transform"
|
||||
BackendTraceModelLoad BackendTraceType = "model_load"
|
||||
BackendTraceScore BackendTraceType = "score"
|
||||
BackendTraceTokenClassify BackendTraceType = "token_classify"
|
||||
BackendTracePatternPII BackendTraceType = "pattern_pii"
|
||||
BackendTraceVectorStore BackendTraceType = "vector_store"
|
||||
)
|
||||
|
||||
@@ -59,10 +61,12 @@ type BackendTrace struct {
|
||||
// runaway buffer when a caller streams MB-scale payloads.
|
||||
const MaxTraceBodyBytes = 1 << 20
|
||||
|
||||
var backendTraceBuffer *circularbuffer.Queue[*BackendTrace]
|
||||
var backendMu sync.Mutex
|
||||
var backendLogChan = make(chan *BackendTrace, 100)
|
||||
var backendInitOnce sync.Once
|
||||
var (
|
||||
backendTraceBuffer *circularbuffer.Queue[*BackendTrace]
|
||||
backendMu sync.Mutex
|
||||
backendLogChan = make(chan *BackendTrace, 100)
|
||||
backendInitOnce sync.Once
|
||||
)
|
||||
|
||||
// backendMaxBodyBytes caps each captured string value in a BackendTrace.Data
|
||||
// field to keep the /api/backend-traces JSON small enough for the admin UI to
|
||||
|
||||
Reference in New Issue
Block a user