mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 10:28:43 -04:00
c5dcb00b00
engine_runtime links sentencepiece, whose default SPM_PROTOBUF_PROVIDER builds the protobuf-lite 3.14.0 sources it vendors. The generated backend.pb.cc is built against the toolchain's protobuf 3.21.12. Both ended up in the binary: 476 google::protobuf:: symbols came from the archive, 278 of them also defined by libprotobuf.so, and the archive won, because once ld pulls a member in for sentencepiece's own code every reference binds to the definitions that member carries. The visible symptom is one function. ParseContext::ParseMessage(MessageLite*, const char*) is what a generated _InternalParse calls for a submessage field and for nothing else, so flat messages parsed and nested ones did not: a TranscriptResult carrying segments serialized to correct bytes that the same process could not read back, and TranscriptLiveRequest, a oneof of submessages, could not have been parsed at all. Underneath that, 3.21 generated code was running 3.14 arena, ArenaStringPtr and ExtensionSet code. -Wl,--exclude-libs does not fix it. It makes those symbols LOCAL in .dynsym and the parse still fails, because the binding was decided at static link time and no visibility flag revisits it. Setting SPM_PROTOBUF_PROVIDER to "package" before add_subdirectory points sentencepiece at the protobuf the generated code was already built against. Zero google::protobuf:: definitions remain in the executable afterwards, every nested message round trips, and citrinet_asr, which parses a SentencePiece ModelProto at load time and would break first if this were wrong, still tokenizes and transcribes correctly. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>