fix(cluster): stop a late request frame reading as the worker's verdict

Making a worker's refusal reaping evidence created a defect one layer
along, at the producer. The worker refused a ReadStreamRequest failure with
ErrStreamRequestInvalid and its own comment said "Includes the deadline
above expiring", which was harmless while every refusal reached the
frontend as "no route" and became a reap the moment one of them did not. So
a request frame that had merely not ARRIVED yet was reported as a
non-transient verdict about a backend.

It is reachable on the relay path, which carries most production traffic:
the worker's header timer starts when the OWNING replica opens the stream,
while the frame is written by the DIALLING replica only after the relay's
acceptance travels back to it, so a whole peer-link round trip runs inside
that window, on a link this design deliberately loads with multi-gigabyte
artifacts beside token streams. For a long-deadline caller the endpoint is
ConnectionEvictingClient, which stops the model across the fleet. It also
falsified the "neither clears on its own" argument that licensed the reap.

There is now a fourth refusal, ErrStreamNotServed, for what a worker could
not serve for a reason of its OWN. It is deliberately outside
IsWorkerAnswer, so it reaches a consumer under the no-route umbrella and
reaps nothing, which is the same treatment an unrecognised code already
gets. Four producers move onto it: a request frame that timed out (a
malformed one stays a verdict, because that is a frontend bug no retry
fixes), both SetReadDeadline failures, which are facts about the stream and
not about a target nothing has dialled yet, and WriteStreamRefusal's
default for a reason nobody classified.

classifyServiceFailure keeps ErrStreamTargetUnavailable as its default on
purpose: inverting it would make errno enumeration the single point of
failure for the reap, and a miss there is a row nothing can ever delete.
What it gains is a deny-list of two causes that are provably this worker's
own clock or its own context.

Also:

- The read-site caller-deadline guard in the handshake was unpinned: the
  existing seam spends the budget before the handshake starts, so only the
  write could ever fail. A spec whose deadline falls between the request and
  the reply pins it, and each guard now reddens on its own.
- The documented worker-first failure line omitted the JSON error envelope
  the old frontend returns, so an operator grepping it found nothing.
- The peer-link disclosure names the aimable per-session receive window in
  all four places, and LastDialErrorOf records why a third consumer must go
  through IsWorkerAnswer rather than roll its own list.

Assisted-by: Claude Opus 5 [claude-code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

pkg/grpc/backend.go

@@ -133,6 +133,20 @@ const maxBackendUnwrapDepth = 16
 // pkg/model is an *InFlightTrackingClient over a *FileStagingClient over the
 // real one, so both callers were asking a wrapper that had no answer and
 // reading nil as "the transport was fine".
+//
+// WHAT IT ANSWERS IS NOT "was this the transport's fault". It answers "what did
+// the dialler last return", and in distributed mode some of those values are a
+// WORKER'S OWN REFUSAL, which means the tunnel worked and the worker spoke.
+// Telling those apart is cluster.IsWorkerAnswer, and the two production callers
+// (nodes.unroutable, model.transportFailure) both go through it. A new caller
+// that matches on sentinels of its own would be re-creating the collapse this
+// phase spent two rounds removing: the reap guards and the dialler would stop
+// agreeing on which errors are evidence.
+//
+// Nothing structural prevents that, unlike the WrappedBackend rule in
+// hack/lint/ which makes decorator transparency impossible to forget. With two
+// callers, both funnelling through one predicate, a ruleguard rule is not worth
+// its false positives; if a third appears, it is. Recorded as a phase-3 note.
 func LastDialErrorOf(b Backend) error {
 	for range maxBackendUnwrapDepth {
 		if b == nil {