fix(distributed): reject wrong-model requests on the remaining modalities (#10990)

#10970 gave the four PredictOptions RPCs a model-identity check so a
backend reached through a stale distributed route rejects the request
instead of answering from whatever model it holds (#10952). Every other
modality shares that exposure: the route is cached by host:port, a worker
can recycle a stopped backend's port for another model's backend, and a
liveness-only probe cannot tell a stale row from a valid one.

Extends the same mechanism to the 21 remaining request messages that reach
a backend through the router, using the pattern #10970 established rather
than a parallel one:

- proto: ModelIdentity on each modality request message.
- controller: populated from ModelConfig.Model at the call site that also
  builds ModelOptions, so load-time and request-time values are equal by
  construction.
- backends: one generic guard in pkg/grpc/server.go (27 Go backends), the
  method set in backend/python/common (36 Python backends), llama-cpp
  (AudioTranscription/Stream, Rerank, Score) and privacy-filter
  (TokenClassify).
- reconcile already drops the stale row on IsModelMismatch; no change.

TTSRequest and SoundGenerationRequest get a SEPARATE ModelIdentity field
rather than reusing their existing `model`: FileStagingClient rewrites
`model` to a worker-local path, so comparing it would reject valid
requests in exactly the configuration this guards.

AudioEncode/AudioDecode are deliberately left unguarded: the opus codec
backend is loaded from a literal rather than a ModelConfig, so no value
carries the equality guarantee the comparison depends on. The four
bidirectional stream RPCs are out of scope; they bypass reconcile.

Empty means skip on both sides, so an old controller, an old backend, and
the bare request structs in tests/e2e-backends all keep working.


Assisted-by: Claude Code:claude-opus-4-8 [Read] [Edit] [Bash]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

backend/cpp/llama-cpp/grpc-server.cpp

@@ -1416,7 +1416,11 @@ public:
     // field and for the synthetic PredictOptions this server builds internally
     // for ASR, and the loaded side is empty when such a controller performed
     // the load. A false rejection is worse than the miss it prevents.
-    grpc::Status checkModelIdentity(const backend::PredictOptions* request) {
+    // Templated over the request type: every guarded request message exposes
+    // modelidentity(), and one body keeps the rule identical across modalities
+    // rather than repeating it per RPC.
+    template <typename Request>
+    grpc::Status checkModelIdentity(const Request* request) {
         if (request == nullptr || request->modelidentity().empty()) {
             return grpc::Status::OK;
         }
@@ -2846,6 +2850,8 @@ public:
     }
 
     grpc::Status Rerank(ServerContext* context, const backend::RerankRequest* request, backend::RerankResult* rerankResult) override {
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
         if (!params_base.embedding || params_base.pooling_type != LLAMA_POOLING_TYPE_RANK) {
             return grpc::Status(grpc::StatusCode::UNIMPLEMENTED, "This server does not support reranking. Start it with `--reranking` and without `--embedding`");
         }
@@ -2970,6 +2976,8 @@ public:
     grpc::Status Score(ServerContext* context, const backend::ScoreRequest* request, backend::ScoreResponse* response) override {
         auto auth = checkAuth(context);
         if (!auth.ok()) return auth;
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
         if (params_base.model.path.empty()) {
             return grpc::Status(grpc::StatusCode::FAILED_PRECONDITION, "Model not loaded");
         }
@@ -3428,6 +3436,8 @@ public:
                                     backend::TranscriptResult* response) override {
         auto auth = checkAuth(context);
         if (!auth.ok()) return auth;
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
 
         backend::Reply reply;
         grpc::Status st = runTranscriptionAsCompletion(context, request, &reply);
@@ -3446,6 +3456,8 @@ public:
                                           grpc::ServerWriter<backend::TranscriptStreamResponse>* writer) override {
         auto auth = checkAuth(context);
         if (!auth.ok()) return auth;
+        auto identity = checkModelIdentity(request);
+        if (!identity.ok()) return identity;
 
         // Buffered streaming: run the transcription as a normal chat
         // completion, then emit one delta + one final event. Real

backend/cpp/privacy-filter/grpc-server.cpp

@@ -41,6 +41,11 @@ namespace {
 // per loaded model. g_mu guards (re)load against in-flight classification.
 std::mutex          g_mu;
 pf_ctx *            g_ctx = nullptr;
+// The ModelOptions.Model this process loaded, compared against
+// TokenClassifyRequest.ModelIdentity so a request that arrived through a stale
+// distributed route is rejected rather than answered from the wrong model
+// (#10952). Guarded by g_mu like the rest of the engine state.
+std::string         g_loaded_model_identity;
 std::atomic<Server *> g_server{nullptr};
 
 // Resolve the device string the engine expects ("cpu" / "gpu" / "cuda" /
@@ -113,17 +118,48 @@ public:
         }
 
         g_ctx = ctx;
+        // Record what we loaded so TokenClassify can reject a request meant
+        // for a different model. request->model(), not modelfile(): it is the
+        // value the controller also sends as ModelIdentity, and the two are
+        // read from the same ModelConfig.Model (#10952).
+        g_loaded_model_identity = request->model();
         result->set_success(true);
         result->set_message("privacy-filter loaded (" + device + ")");
         return GStatus::OK;
     }
 
+    // checkModelIdentity mirrors pkg/grpc/server.go,
+    // backend/python/common/model_identity.py and the llama-cpp server. In
+    // distributed mode a worker can recycle a stopped backend's gRPC port for
+    // another model's backend, and the controller's liveness-only probe cannot
+    // tell a stale cached route from a valid one, so the backend has to catch
+    // it. Either side empty means "skip": the request side is empty for a
+    // controller that predates the field, the loaded side when such a
+    // controller performed the load. A false rejection is worse than the miss.
+    // Callers must already hold g_mu.
+    GStatus checkModelIdentity(const backend::TokenClassifyRequest * request) {
+        if (request == nullptr || request->modelidentity().empty()) {
+            return GStatus::OK;
+        }
+        if (g_loaded_model_identity.empty() ||
+            g_loaded_model_identity == request->modelidentity()) {
+            return GStatus::OK;
+        }
+        // NOT_FOUND plus this exact sentinel is the cross-language contract
+        // the router matches on (grpcerrors.ModelMismatchSentinel).
+        return GStatus(StatusCode::NOT_FOUND,
+                       "privacy-filter: model identity mismatch: loaded \"" +
+                       g_loaded_model_identity + "\", requested \"" +
+                       request->modelidentity() + "\"");
+    }
+
     GStatus TokenClassify(ServerContext *, const backend::TokenClassifyRequest * request,
                           backend::TokenClassifyResponse * response) override {
         std::lock_guard<std::mutex> lock(g_mu);
         if (!g_ctx) {
             return GStatus(StatusCode::FAILED_PRECONDITION, "Model not loaded");
         }
+        if (GStatus id = checkModelIdentity(request); !id.ok()) return id;
 
         const std::string & text = request->text();
         if (text.empty()) {