fix(router): production-ready request router + auto-size batch for embedding/rerank (#10104)

* fix(router): score classifier production-readiness

Conversation trimming runs through the classifier model's chat template
and trims by exact token count, sized to the model's n_batch which is
now scaled to context so long probes can't crash the backend. Missing
chat_message templates are a hard error at router build time. Router-
facing factories (Embedder/Scorer/Reranker/TokenCounter) re-resolve
ModelConfig per call so a model installed post-startup doesn't bind a
stub Backend="" config and silently fall into the loader's auto-
iterate path.

New 'vector_store' backend trace recorded inside localVectorStore on
every Search/Insert — including the backend-load-failure path that
previously vanished into an xlog.Warn — with outcome tagging
(hit/miss/empty_store/backend_load_error/find_error/insert_error/ok).
Companion cleanup drops misleading similarity:0 and input_tokens_count:0
from non-hit and text-mode traces.

Gallery local-store-development aliases to 'local-store' so the master
image satisfies pkg/model.LocalStoreBackend lookups from the embedding
cache.

Misc: llama-cpp TokenizeString reads the correct 'prompt' JSON key
(the original bug); ModelTokenize nil-guard; non-fatal mitm proxy
startup; PII 'route_local' renamed to 'allow' with docs/UI in sync;
model-editor footer no longer eats the edit area on small screens;
several config-editor template/dropdown/section fixes.

Tests: e2e router specs (casual/code-hint + long-conversation trim),
vector_store trace specs, lazy-factory specs, gallery dev-alias
resolution, Playwright trace badge + scroll regression.

Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(backend): auto-size batch to context for embedding and rerank models

Embedding and rerank models pool over the whole input in a single physical batch (n_ubatch). With batch left at the 512 default, the backend rejects longer inputs with "input is too large to process", silently capping a large-context embedder (e.g. 8k/32k) at 512 tokens. Size n_batch to the context for these single-pass usecases, mirroring the existing FLAG_SCORE behaviour; an explicit batch: still wins.

Extracts EffectiveContextSize/EffectiveBatchSize from grpcModelOpts so the effective decode window has one home for other callers to reuse.

Adds an e2e-aio regression test that embeds a >512-token input. The AIO embedding model is switched to nomic-embed-text-v1.5 (2048 context) because the previous granite model was capped at 512 tokens and could not exercise the larger batch.

Assisted-by: claude-code:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(gallery): raise arch-router scoring output cap via parallel:64

Scoring decodes the whole prompt+candidate in a single llama_decode and
reads one logit row per candidate token. The vendored llama.cpp server
caps causal output rows at n_parallel, so the default of 1 aborts with
GGML_ASSERT(n_outputs_max <= cparams.n_outputs_max) on multi-token route
labels. Set options: [parallel:64] on both arch-router quant entries to
lift the cap; kv_unified (the grpc-server default) keeps the full context
per sequence, so this does not split the KV cache.

Assisted-by: claude-code:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>

pkg/mcp/localaitools/dto.go

@@ -188,7 +188,7 @@ type UsageBucket struct {
 type PIIPattern struct {
 	ID             string `json:"id"`
 	Description    string `json:"description"`
-	Action         string `json:"action"` // mask | block | route_local
+	Action         string `json:"action"` // mask | block | allow
 	MaxMatchLength int    `json:"max_match_length"`
 }
 
@@ -222,12 +222,12 @@ type PIIRedactTestRequest struct {
 
 // PIIRedactTestResult is the output for test_pii_redaction. spans
 // describes where the redactor matched; redacted is the text after
-// applying mask actions; blocked / local_only flag stronger actions.
+// applying mask actions; blocked / masked flag what was done.
 type PIIRedactTestResult struct {
-	Redacted  string        `json:"redacted"`
-	Spans     []PIIEventSpan `json:"spans"`
-	Blocked   bool          `json:"blocked"`
-	LocalOnly bool          `json:"local_only"`
+	Redacted string         `json:"redacted"`
+	Spans    []PIIEventSpan `json:"spans"`
+	Blocked  bool           `json:"blocked"`
+	Masked   bool           `json:"masked"`
 }
 
 type PIIEventSpan struct {
@@ -243,7 +243,7 @@ type PIIEventSpan struct {
 // to runtime_settings.json so the next start re-applies them.
 type PIIPatternActionUpdate struct {
 	ID       string `json:"id" jsonschema:"Pattern id to mutate (e.g. email, ssn, credit_card, api_key_prefix)."`
-	Action   string `json:"action,omitempty" jsonschema:"New action: mask, block, or route_local. Optional — omit to leave the action unchanged."`
+	Action   string `json:"action,omitempty" jsonschema:"New action: mask, block, or allow. Optional — omit to leave the action unchanged."`
 	Disabled *bool  `json:"disabled,omitempty" jsonschema:"Set true to skip this pattern entirely; false to re-enable. Optional — omit to leave enabled-state unchanged."`
 }
 

pkg/mcp/localaitools/inproc/client.go

@@ -823,9 +823,9 @@ func (c *Client) TestPIIRedaction(_ context.Context, req localaitools.PIIRedactT
 	}
 	res := c.PIIRedactor.Redact(req.Text)
 	out := &localaitools.PIIRedactTestResult{
-		Redacted:  res.Redacted,
-		Blocked:   res.Blocked,
-		LocalOnly: res.LocalOnly,
+		Redacted: res.Redacted,
+		Blocked:  res.Blocked,
+		Masked:   res.Masked,
 	}
 	for _, s := range res.Spans {
 		out.Spans = append(out.Spans, localaitools.PIIEventSpan{

pkg/mcp/localaitools/tools_middleware.go

@@ -47,13 +47,13 @@ func registerMiddlewareTools(s *mcp.Server, client LocalAIClient, opts Options)
 
 	mcp.AddTool(s, &mcp.Tool{
 		Name:        ToolSetPIIPatternAction,
-		Description: "Change a PII pattern's action (mask|block|route_local) and/or disabled state in-process. TRANSIENT: the mutation is lost on restart unless followed by persist_pii_patterns. Admin-required.",
+		Description: "Change a PII pattern's action (mask|block|allow) and/or disabled state in-process. TRANSIENT: the mutation is lost on restart unless followed by persist_pii_patterns. Admin-required.",
 	}, func(ctx context.Context, _ *mcp.CallToolRequest, args PIIPatternActionUpdate) (*mcp.CallToolResult, any, error) {
 		if args.ID == "" {
 			return errorResultf("id is required"), nil, nil
 		}
 		if args.Action == "" && args.Disabled == nil {
-			return errorResultf("at least one of action (mask, block, route_local) or disabled must be set"), nil, nil
+			return errorResultf("at least one of action (mask, block, allow) or disabled must be set"), nil, nil
 		}
 		if err := client.SetPIIPatternAction(ctx, args); err != nil {
 			return errorResult(err), nil, nil

pkg/mcp/localaitools/tools_pii.go

@@ -9,7 +9,7 @@ import (
 func registerPIITools(s *mcp.Server, client LocalAIClient, _ Options) {
 	mcp.AddTool(s, &mcp.Tool{
 		Name:        ToolListPIIPatterns,
-		Description: "List the active PII regex pattern set. Each entry shows the pattern id, description, and current action (mask, block, route_local). Read-only.",
+		Description: "List the active PII regex pattern set. Each entry shows the pattern id, description, and current action (mask, block, allow). Read-only.",
 	}, func(ctx context.Context, _ *mcp.CallToolRequest, _ struct{}) (*mcp.CallToolResult, any, error) {
 		patterns, err := client.ListPIIPatterns(ctx)
 		if err != nil {