Compare commits

...

1 Commits

Author SHA1 Message Date
Ettore Di Giacinto
83110891fd fix(go-grpc-server): always close resultChan
By not closing the channel, if a server not implementing PredictStream
receives a client call would hang indefinetly as would wait for
resultChan to be consumed.

If the prediction stream returns we close the channel now and we wait
for the goroutine to finish.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-10-05 00:07:58 +02:00

View File

@@ -144,6 +144,8 @@ func (s *server) PredictStream(in *pb.PredictOptions, stream pb.Backend_PredictS
}()
err := s.llm.PredictStream(in, resultChan)
// close the channel, so if resultChan is not closed by the LLM (maybe because does not implement PredictStream), the client will not hang
close(resultChan)
<-done
return err