walcz-de 6d1bbb74c4 fix(backend/python): don't await sync servicer behaviors in AsyncModelIdentityInterceptor (#10980) ...

* fix(backend/python): don't await sync servicer behaviors in AsyncModelIdentityInterceptor

The model-identity interceptor (added for #10952) is installed on every Python
backend's gRPC server. Its grpc.aio variant invokes the wrapped servicer
behavior itself and awaits the result unconditionally:

    result = await original(request, context)                 # LoadModel
    return await original_unary(request, context)             # guarded RPCs
    async for response in original_stream(request, context):  # streaming

But a backend's servicer methods may be plain sync functions. The transformers
backend, for one, defines `def LoadModel` and `def Embedding` (not `async def`).
grpc.aio's own dispatch adapts both shapes, but this interceptor calls the
behavior directly and bypasses that. For a sync method `original(...)` returns a
message object, not a coroutine, so the `await` raises:

    TypeError: object Result can't be used in 'await' expression

The model loads, then the LoadModel RPC dies on return; the guarded sync
Embedding fails the same way. It happens on every platform, not just one backend
build. CI never caught it because AsyncModelIdentityInterceptor had no
behavioral test -- only an "is it installed" assertion.

Fix: await only when the behavior actually returned an awaitable
(inspect.isawaitable), mirroring grpc.aio's own sync/async adaptation. The
streaming guard iterates a sync generator with `for` and an async one with
`async for`.

Adds async-path coverage to model_identity_test.py exercising both sync and
async LoadModel / guarded-unary / streaming behaviors. The sync cases fail on
the current code with the TypeError above and pass with this fix.

Signed-off-by: stefanwalcz <stefan.walcz@walcz.de>

* fix(backend/python): dispatch sync servicer behaviors off the event loop

Addresses review feedback: awaiting only awaitable results removed the
TypeError, but still ran a sync LoadModel/Embedding -- and stepped a sync stream
via next() -- on the asyncio event-loop thread, so a slow load/inference/stream
could freeze all aio RPC handling.

Route sync behavior through run_in_executor (a worker thread) while awaiting
native async behavior directly. A callable wrapper that returns an awaitable is
run in the thread and its awaitable awaited back on the loop. Sync streaming
pulls each item via the executor with a done sentinel, so StopIteration cannot
escape through a Future.

Adds regression tests that record the handler thread id and assert it differs
from the event-loop thread, for LoadModel, a guarded unary RPC and a sync stream.

Signed-off-by: stefanwalcz <stefan.walcz@walcz.de>

---------

Signed-off-by: stefanwalcz <stefan.walcz@walcz.de>

2026-07-22 16:03:46 +02:00

11 KiB

Raw Blame History

View Raw