mirror of
https://github.com/mudler/LocalAI.git
synced 2026-11-06 04:23:15 -05:00
feat(addons): move connector file bytes by handle
A connector add-on that declares files.transfer no longer carries file bytes in the connector envelope. The host puts a random handle in the request of the op media (write) or send_file (read), and the add-on streams the file with two new host operations, connector.file.put and connector.file.get, in chunks of at most 256 KiB. The process that runs the add-on registers the handle when the call arrives and ends it when the call returns. A handle is bound to the add-on, so another add-on gets the same answer as for an unknown handle. An unfinished inbound file is dropped with its partial bytes, and a handle with no call for two minutes expires. Inbound bytes go to the blob store of the workspace, outbound bytes come from it, so a cluster needs the shared object store and no shared disk. The file is never held in memory. The host overwrites the size and hash of an inbound file with its own record. The change is additive: the field files.transfer needs manifest API 2, a kind without it keeps the inline path and its 1 MiB cap, and the host never sends it a handle. A call that carries a handle may run up to 15 minutes. The ceiling of a file also stays under the largest file of a workspace. Assisted-by: Claude:claude-sonnet-5-5 go test
This commit is contained in:
21 files changed
+1453
-14
No files matched your search
@@ -26,6 +26,7 @@ import (
|
||||
"github.com/mudler/LocalAI/core/services/addons/backendlib"
|
||||
"github.com/mudler/LocalAI/core/services/addons/catalog"
|
||||
"github.com/mudler/LocalAI/core/services/addons/gateway"
|
||||
"github.com/mudler/LocalAI/core/services/addons/hostops"
|
||||
"github.com/mudler/LocalAI/core/services/addons/hostsettings"
|
||||
"github.com/mudler/LocalAI/core/services/addons/packages"
|
||||
"github.com/mudler/LocalAI/core/services/addons/pins"
|
||||
@@ -113,9 +114,12 @@ type Application struct {
|
||||
backendLibrary *backendlib.Installer
|
||||
addonManager *addons.Manager
|
||||
addonHostOps *addonruntime.HostOps
|
||||
addonDataRoot string // parent of the private data directory of each add-on; empty when none runs here
|
||||
addonsProbe addonsProbeState
|
||||
modelsProbe modelsProbeState
|
||||
// addonTransfers holds the handles of connector file transfers; see transfers.
|
||||
addonTransfers *hostops.Transfers
|
||||
addonTransfersOnce sync.Once
|
||||
addonDataRoot string // parent of the private data directory of each add-on; empty when none runs here
|
||||
addonsProbe addonsProbeState
|
||||
modelsProbe modelsProbeState
|
||||
|
||||
// libraryStore is the library sources file when no add-on owner runs here.
|
||||
libraryStore *catalog.Sources
|
||||
|
||||
Reference in new issue
Block a user