Files
LocalAI/go.mod
mudler's LocalAI [bot] f735cb24c0 fix(worker): reap deleted backends and stop models that live on a worker (#10956)
* fix(worker): reap deleted backends and stop models that live on a worker

Three related backend-lifecycle defects, all reachable from the same
production incident on a Jetson/Thor worker: a deleted backend's gRPC
process survived ~40 minutes with its directory removed from disk, a later
model load was routed to that orphan and failed with a certifi path pointing
into the deleted directory, and the admin could not stop the model because
the frontend reported it as not loaded.

1. backend.delete orphaned the process it claimed to delete
------------------------------------------------------------
s.processes is keyed by `modelID#replicaIndex` (buildProcessKey), so the
backend name never appeared in a key and was recorded nowhere on the
process. backend.delete resolved its target via isRunning/stopBackend, whose
prefix path only matches a bare *modelID* - a delete keyed on a backend name
resolved to zero keys, the stop silently no-op'd, and the files were removed
out from under a live process.

The install fast path then handed that orphan back out: it returns any live
process for the (model, replica) slot without checking which backend started
it, so a reinstalled variant inherited the deleted backend's port.

- Record backendName on backendProcess, threaded installBackend ->
  startBackend.
- Add resolveProcessKeysForBackend, matching the recorded name and resolving
  alias <-> concrete via ListSystemBackends *before* DeleteBackendFromSystem
  erases the metadata that carries the alias. Alias resolution failure
  degrades to name-only matching so a delete never fails on it.
- backend.stop goes through resolveStopTargets, which accepts a backend
  name, a model name, or an exact modelID#replica key. Its payload field is
  named "backend" but is published with all three meanings: the admin UI
  sends a backend name, UnloadRemoteModel sends a model name, and the
  router's abandoned-load reap (#10948) sends an exact replica key.
  Narrowing it to backend names alone would strand the latter two.
  backend.delete stays strict - its identifier is unambiguously a backend.
- Gate the install fast path on processMatchesBackend so a slot held by a
  different backend is restarted rather than reused. Processes with no
  recorded name (pre-upgrade) are accepted, so rollout does not restart
  every running backend.
- stopBackendExact reports a real stop failure - the process still being
  alive afterwards, which is precisely what finishBackendStop already
  detects to keep the entry and its port reserved - and backend.delete no
  longer replies success when it knew about a process and could not kill it.
  "No process was running" stays a success but is logged, so the orphan case
  is visible rather than silent.

2. /backend/shutdown reported a running model as missing
---------------------------------------------------------
ModelLoader.deleteProcess short-circuits on a miss in this replica's
in-memory store. In distributed mode the authoritative record of "is this
model loaded" is the shared node registry: a frontend replica that never
served the model itself (load balancer picked a peer, or the replica
restarted) has no local entry. The remote unload path that pkg/model
documents ("when ShutdownModel is called for a model with no local process,
UnloadRemoteModel is called") sat behind that short-circuit, unreachable in
exactly the case it exists for. #10865 reworked this function but kept the
short-circuit at the top, so the gap survived that refactor.

- deleteProcess consults the remote unloader on a local-store miss, via a
  shared unloadRemote helper so this branch and the existing
  no-local-process branch both prefer #10865's RemoteModelContextUnloader,
  preserving force propagation across the distributed boundary.
- UnloadRemoteModelContext reports ErrRemoteModelNotLoaded when no node has
  the model; it previously returned nil, making a no-op stop
  indistinguishable from a real one. The converse case (nodes have it, none
  could be stopped) already errors since #10865 joined the per-node
  failures, so that half of the original fix was dropped as redundant.
- Only when the model is absent locally AND cluster-wide does the endpoint
  report not-found, now 404 naming both scopes rather than a bare 500.
- modelNotFoundErr becomes the exported ErrModelNotFound so the HTTP layer
  can map it without string matching; watchdog's identity comparison becomes
  errors.Is.

3. Coverage for the bounded Free() that #10865 shipped untested
----------------------------------------------------------------
The original branch also bounded the pre-stop Free(), but #10865 landed that
fix first (workerBackendFreeTimeout, applied in both stopBackendExact and
handleModelUnload). That production change is therefore DROPPED here as
superseded - master's version is strictly better, since it also releases the
supervisor mutex across the call and keeps the port reserved until
termination completes.

What #10865 did not ship is a test, and the bound is load-bearing: the
router-side reap in #10948 sends backend.stop for an abandoned load, and
against a wedged backend an unbounded Free would swallow that stop before it
reached the process. Nothing failed if the bound regressed.

The spec stands up a real gRPC backend server whose Free handler never
returns - what a Python backend looks like when its single worker thread
(PYTHON_GRPC_MAX_WORKERS=1 on 37 backends) is occupied by a stuck LoadModel.
A stub socket is not sufficient and was tried first: without a completed
HTTP/2 handshake, gRPC's own ~20s connect timeout ends the call, so that
version passed against the very bug it targets. With the connection READY,
only the caller's deadline can end it, so the spec hangs to its 60s limit if
the timeout is removed and passes with it.

Its fixture process is deliberately never started. go-processmanager v0.1.1
writes Process.pid from readPID() without synchronization, so a live process
races its own monitor goroutine under -race - reproducible with a bare
Run()+Stop() and unrelated to this spec. Since
scripts/model-lifecycle-conformance.sh runs this package with -race and is
fail-closed, starting one would turn that gate red on an upstream defect. An
unstarted process still proves the point: the stop is reached and the slot
released, which is exactly what an unbounded Free prevents.

Verified: make lint (new-from-merge-base origin/master) reports 0 issues;
scripts/model-lifecycle-conformance.sh passes all three stages including the
FizzBee liveness check (1458 states, IsLive: true).

Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(distributed): keep remote unload idempotent, ask presence separately

2035a4d25 made UnloadRemoteModel return ErrRemoteModelNotLoaded when no node
holds the model, so ShutdownModel could answer 404 instead of a misleading
500. That narrowed a shared adapter contract to serve one caller and broke
the documented idempotent-unload guarantee, which CI caught on PR #10956:

  [FAIL] Node Backend Lifecycle (NATS-driven) > NATS backend.stop events
         should be no-op for models not on any node [Distributed]
         Expected success, but got: model not loaded on any node

The spec name states the contract outright. The matching unit assertion was
updated in that commit; this e2e one was missed because it lives under
tests/e2e/ with no build tags and does not run in package-scoped test runs.

Caller audit - who breaks when an idempotent unload becomes an error:

- pkg/model/watchdog.go:902 (LRU memory reclaimer) is the serious one. It
  untracks a model ONLY when shutdown returns nil or ErrModelNotFound. A new
  error type means the model is never untracked, so the reclaimer keeps
  re-selecting the same entry and never reclaims - a live wedge whenever a
  local store entry outlives the remote model.
- core/services/galleryop/managers_local.go:43 (DeleteModel) would warn on
  every deletion of an already-unloaded model.
- core/services/modeladmin/{state,config,remote_sync}.go stop instances
  best-effort against models that are frequently not loaded.
- deleteProcess itself: the no-local-process branch returns the unload result
  directly, so a stale local entry for a model no longer on any node turned a
  previously-successful cleanup into a failure.

Only ShutdownModel wants the distinction, and only on the local-store-miss
path. So the distinction moves to the caller instead of the contract:

- UnloadRemoteModel/UnloadRemoteModelContext return nil again when no node
  has the model, and ErrRemoteModelNotLoaded is removed.
- New optional RemoteModelPresenceChecker (HasRemoteModel) answers the
  question directly. deleteProcess consults it BEFORE unloading, because an
  idempotent unload cannot report afterwards whether anything was stopped.
  Absent locally AND cluster-wide is the only case that reports 404.
- A failed registry lookup is surfaced rather than reported as absence: an
  unreachable registry is not evidence a model is gone, and answering a
  confident 404 off a failed lookup is how an operator gets told a running
  model does not exist.
- Unloaders that predate the extension keep working - deleteProcess attempts
  the unload rather than refusing it - and compile-time assertions in the
  nodes package now pin all three optional interfaces, since both are
  consumed by runtime type assertion where drift degrades behavior silently
  instead of failing the build.

The contract is now pinned at both levels that disagreed, each spec pointing
at the other: "with no nodes returns nil" in unloader_test.go and "should be
no-op for models not on any node" in node_lifecycle_test.go.

Verified: full distributed e2e suite 233 passed / 0 failed (the suite that
failed 232/1 in CI); pkg/model and core/services/nodes green; make lint
new-from-merge-base reports 0 issues.

Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(distributed): drop replica rows when a worker stops a backend

A worker returns a stopped backend's gRPC port to its allocator as soon as
the process is confirmed dead, and hands it to the next backend that starts.
The controller's NodeModel row for the old address survives, and both
SmartRouter.probeHealth and the HealthMonitor per-model probe verify
liveness, not identity, so once an unrelated backend binds the recycled port
the stale row passes every check and the request is served by the wrong
backend instead of failing.

backend.delete is newly able to trigger this: before #10956 a delete never
actually stopped a process, so it never recycled a port. backend.upgrade has
the identical gap and always did — upgradeBackend force-stops every process
using the binary and starts none back up, while
DistributedBackendManager.UpgradeBackend never removes rows. model.unload is
the one path that gets this right today: it calls RemoveAllNodeModelReplicas
straight after StopBackend.

Report the process keys the worker terminated on the delete and upgrade
replies, and drop the matching rows in RemoteUnloaderAdapter, which already
holds a ModelLocator with RemoveNodeModel. All three call sites funnel
through that adapter, so no new interface, DB migration, or proto change is
needed. A key is reported only once its process is confirmed gone, so the
list stays trustworthy on the partial-failure replies too.

Old workers never populate the new fields. ReportsStoppedProcesses tells
"stopped nothing" apart from "does not report", so an old worker's silence
falls back to the pre-existing probe-based staleness recovery instead of
being mistaken for a completed cleanup.

Quarantine released ports for a short window as an interlock covering the
NATS round-trip between the worker freeing the port and the controller
dropping the row. It is deliberately not derived from HealthCheckInterval:
that cadence is operator-tunable and the per-model reaper can be disabled
outright, so coupling a worker-local constant to it would be silently wrong
on some clusters. Eager row removal is the fix; the delay only closes the
handoff gap.

Identity verification in probeHealth was considered and rejected: Health and
Status carry no backend identity, so it needs a proto change plus an
implementation in 36 Python and 4 C++ Health servicers, it is fail-open for
any backend not yet rebuilt, and the probeCache short-circuit means it would
not even execute during the 30s window where the misroute happens.

Fixes #10952
Refs #10954, #10956

Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(deps): bump go-processmanager, assert real backend termination

go-processmanager wrote Process.PID from readPID() with no synchronization
while its own monitor goroutine cleared the same field on exit, so a bare
Run()+Stop() tripped the race detector without any concurrent access from
the caller. LocalAI hit this on every backend stop.

Upstream fixed it in a94e2b7 by guarding PID with a mutex and adding
CurrentPID() as a race-safe accessor. The exported field was kept to avoid
a breaking change but is now deprecated: a direct read still races the
monitor. No tag carries the fix yet, so pin the pseudo-version.

GetGRPCPID reads through CurrentPID() instead of the field. The accessor
returns the same string under an RLock, so the empty-PID and strconv error
paths are unchanged; it is the only direct field read in the tree.

With the race gone, the Free-timeout spec no longer has to leave its
fixture process unstarted. It now runs a real child and asserts the child
genuinely exits, which is exactly what the earlier workaround gave up: the
spec could show the stop was reached and the slot released, but not that
SIGTERM ever landed. Termination is observed through Done(), which closes
only once the library has waited on the child. The pidfile-based liveness
helpers cannot serve here, because Stop() deletes the pidfile while
releasing the handle and so reports "not alive" even if no signal was sent.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-20 01:09:22 +02:00

513 lines
25 KiB
Modula-2

module github.com/mudler/LocalAI
go 1.26.0
require (
dario.cat/mergo v1.0.2
fyne.io/fyne/v2 v2.7.3
github.com/Masterminds/sprig/v3 v3.3.0
github.com/alecthomas/kong v1.15.0
github.com/anthropics/anthropic-sdk-go v1.42.0
github.com/aws/aws-sdk-go-v2 v1.41.7
github.com/aws/aws-sdk-go-v2/config v1.32.16
github.com/aws/aws-sdk-go-v2/credentials v1.19.17
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.1
github.com/charmbracelet/glamour v1.0.0
github.com/containerd/containerd v1.7.31
github.com/coreos/go-oidc/v3 v3.18.0
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8
github.com/ebitengine/purego v0.10.0
github.com/emirpasic/gods/v2 v2.0.0-alpha
github.com/fsnotify/fsnotify v1.10.1
github.com/go-audio/wav v1.1.0
github.com/go-skynet/go-llama.cpp v0.0.0-20240314183750-6a8041ef6b46
github.com/gofrs/flock v0.13.0
github.com/google/go-containerregistry v0.21.6
github.com/google/uuid v1.6.0
github.com/gpustack/gguf-parser-go v0.24.0
github.com/hpcloud/tail v1.0.0
github.com/ipfs/go-log v1.0.5
github.com/jaypipes/ghw v0.24.0
github.com/joho/godotenv v1.5.1
github.com/klauspost/cpuid/v2 v2.3.0
github.com/labstack/echo/v4 v4.15.1
github.com/libp2p/go-libp2p v0.48.0
github.com/lithammer/fuzzysearch v1.1.8
github.com/mholt/archiver/v3 v3.5.1
github.com/microcosm-cc/bluemonday v1.0.27
github.com/modelcontextprotocol/go-sdk v1.5.0
github.com/mudler/cogito v0.10.1-0.20260609212329-bf4010d31047
github.com/mudler/edgevpn v0.34.0
github.com/mudler/go-processmanager v0.1.2-0.20260719202549-a94e2b7e5f4f
github.com/mudler/memory v0.0.0-20260406210934-424c1ecf2cf8
github.com/mudler/xlog v0.0.6
github.com/nats-io/jwt/v2 v2.7.4
github.com/nats-io/nats.go v1.52.0
github.com/nats-io/nkeys v0.4.15
github.com/ollama/ollama v0.20.4
github.com/onsi/ginkgo/v2 v2.29.0
github.com/onsi/gomega v1.41.0
github.com/openai/openai-go/v3 v3.26.0
github.com/otiai10/copy v1.14.1
github.com/otiai10/openaigo v1.7.0
github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5
github.com/prometheus/client_golang v1.23.2
github.com/robfig/cron/v3 v3.0.1
github.com/russross/blackfriday v1.6.0
github.com/sashabaranov/go-openai v1.41.2
github.com/schollz/progressbar/v3 v3.19.0
github.com/shirou/gopsutil/v3 v3.24.5
github.com/sigstore/sigstore-go v1.1.4
github.com/streamer45/silero-vad-go v0.2.1
github.com/swaggo/echo-swagger v1.5.2
github.com/swaggo/swag v1.16.6
github.com/testcontainers/testcontainers-go v0.42.0
github.com/testcontainers/testcontainers-go/modules/nats v0.42.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0
github.com/timbutler/zxcvbn v1.0.4
github.com/yalue/onnxruntime_go v1.11.0
go.opentelemetry.io/otel v1.44.0
go.opentelemetry.io/otel/exporters/prometheus v0.66.0
go.opentelemetry.io/otel/metric v1.44.0
go.opentelemetry.io/otel/sdk/metric v1.44.0
google.golang.org/grpc v1.80.0
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/postgres v1.6.0
gorm.io/driver/sqlite v1.6.0
gorm.io/gorm v1.31.1
oras.land/oras-go/v2 v2.6.0
)
require (
filippo.io/bigmod v0.1.1-0.20260103110540-f8a47775ebe5 // indirect
filippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.24 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.9 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.23 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.11 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.17 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.42.1 // indirect
github.com/aws/smithy-go v1.25.1 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/buger/jsonparser v1.1.2 // indirect
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20241213102144-19d51d7fe467 // indirect
github.com/digitorus/pkcs7 v0.0.0-20230818184609-3a137a874352 // indirect
github.com/digitorus/timestamp v0.0.0-20231217203849-220c5c2851b7 // indirect
github.com/dunglas/httpsfv v1.1.0 // indirect
github.com/filecoin-project/go-clock v0.1.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-openapi/analysis v0.24.1 // indirect
github.com/go-openapi/errors v0.22.4 // indirect
github.com/go-openapi/loads v0.23.2 // indirect
github.com/go-openapi/runtime v0.29.2 // indirect
github.com/go-openapi/strfmt v0.25.0 // indirect
github.com/go-openapi/swag/cmdutils v0.25.4 // indirect
github.com/go-openapi/swag/conv v0.25.4 // indirect
github.com/go-openapi/swag/fileutils v0.25.4 // indirect
github.com/go-openapi/swag/jsonname v0.25.4 // indirect
github.com/go-openapi/swag/jsonutils v0.25.4 // indirect
github.com/go-openapi/swag/loading v0.25.4 // indirect
github.com/go-openapi/swag/mangling v0.25.4 // indirect
github.com/go-openapi/swag/netutils v0.25.4 // indirect
github.com/go-openapi/swag/stringutils v0.25.4 // indirect
github.com/go-openapi/swag/typeutils v0.25.4 // indirect
github.com/go-openapi/swag/yamlutils v0.25.4 // indirect
github.com/go-openapi/validate v0.25.1 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/google/certificate-transparency-go v1.3.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
github.com/in-toto/attestation v1.1.2 // indirect
github.com/in-toto/in-toto-golang v0.9.0 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/jolestar/go-commons-pool/v2 v2.1.2 // indirect
github.com/klippa-app/go-pdfium v1.19.2 // indirect
github.com/mattn/go-sqlite3 v1.14.28 // indirect
github.com/moby/moby/api v1.54.2 // indirect
github.com/moby/moby/client v0.4.1 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.9.1 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/sigstore/protobuf-specs v0.5.1 // indirect
github.com/sigstore/rekor v1.4.3 // indirect
github.com/sigstore/rekor-tiles/v2 v2.0.1 // indirect
github.com/sigstore/sigstore v1.10.0 // indirect
github.com/sigstore/timestamp-authority/v2 v2.0.3 // indirect
github.com/standard-webhooks/standard-webhooks/libraries v0.0.0-20260508151727-1282bb917829 // indirect
github.com/stretchr/testify v1.11.1 // indirect
github.com/sv-tools/openapi v0.2.1 // indirect
github.com/swaggo/swag/v2 v2.0.0-rc4 // indirect
github.com/tetratelabs/wazero v1.11.0 // indirect
github.com/theupdateframework/go-tuf/v2 v2.3.0 // indirect
github.com/tmc/langchaingo v0.1.14 // indirect
github.com/transparency-dev/formats v0.0.0-20251017110053-404c0d5b696c // indirect
github.com/transparency-dev/merkle v0.0.2 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
go.mongodb.org/mongo-driver v1.17.6 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 // indirect
sigs.k8s.io/yaml v1.6.0 // indirect
)
require (
github.com/JohannesKaufmann/dom v0.2.0 // indirect
github.com/JohannesKaufmann/html-to-markdown/v2 v2.4.0 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/PuerkitoBio/goquery v1.10.3 // indirect
github.com/RoaringBitmap/roaring/v2 v2.4.5 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/antchfx/htmlquery v1.3.4 // indirect
github.com/antchfx/xmlquery v1.4.4 // indirect
github.com/antchfx/xpath v1.3.6 // indirect
github.com/bits-and-blooms/bitset v1.24.0 // indirect
github.com/blevesearch/bleve/v2 v2.5.7 // indirect
github.com/blevesearch/bleve_index_api v1.2.11 // indirect
github.com/blevesearch/geo v0.2.4 // indirect
github.com/blevesearch/go-faiss v1.0.26 // indirect
github.com/blevesearch/go-porterstemmer v1.0.3 // indirect
github.com/blevesearch/gtreap v0.1.1 // indirect
github.com/blevesearch/mmap-go v1.0.4 // indirect
github.com/blevesearch/scorch_segment_api/v2 v2.3.13 // indirect
github.com/blevesearch/segment v0.9.1 // indirect
github.com/blevesearch/snowballstem v0.9.0 // indirect
github.com/blevesearch/upsidedown_store_api v1.0.2 // indirect
github.com/blevesearch/vellum v1.1.0 // indirect
github.com/blevesearch/zapx/v11 v11.4.2 // indirect
github.com/blevesearch/zapx/v12 v12.4.2 // indirect
github.com/blevesearch/zapx/v13 v13.4.2 // indirect
github.com/blevesearch/zapx/v14 v14.4.2 // indirect
github.com/blevesearch/zapx/v15 v15.4.2 // indirect
github.com/blevesearch/zapx/v16 v16.2.8 // indirect
github.com/bwmarrin/discordgo v0.29.0 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/cyphar/filepath-securejoin v0.6.1 // indirect
github.com/emersion/go-imap/v2 v2.0.0-beta.5 // indirect
github.com/emersion/go-message v0.18.2 // indirect
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
github.com/emersion/go-smtp v0.24.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/eritikass/githubmarkdownconvertergo v0.1.10 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.9.0 // indirect
github.com/go-git/go-git/v5 v5.19.0 // indirect
github.com/go-telegram/bot v1.17.0 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/gocolly/colly v1.2.0 // indirect
github.com/gofiber/fiber/v2 v2.52.13 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/gomarkdown/markdown v0.0.0-20260411013819-759bbc3e3207 // indirect
github.com/google/go-github/v69 v69.2.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jung-kurt/gofpdf v1.16.2 // indirect
github.com/kennygrant/sanitize v1.2.4 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/mschoch/smat v0.2.0 // indirect
github.com/mudler/LocalAGI v0.0.0-20260606071251-14aed1ae4336
github.com/mudler/localrecall v0.6.3 // indirect
github.com/mudler/skillserver v0.0.7-0.20260520220837-a7317cbf9145
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/oxffaa/gopher-parse-sitemap v0.0.0-20191021113419-005d2eb1def4 // indirect
github.com/philippgille/chromem-go v0.7.0 // indirect
github.com/pion/transport/v4 v4.0.1 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/rs/zerolog v1.31.0 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/slack-go/slack v0.17.3 // indirect
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf // indirect
github.com/swaggo/files/v2 v2.0.2 // indirect
github.com/temoto/robotstxt v1.1.2 // indirect
github.com/thoj/go-ircevent v0.0.0-20210723090443-73e444401d64 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/traefik/yaegi v0.16.1 // indirect
github.com/valyala/fasthttp v1.68.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.mau.fi/util v0.3.0 // indirect
go.starlark.net v0.0.0-20250417143717-f57e51f710eb // indirect
google.golang.org/appengine v1.6.8 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
jaytaylor.com/html2text v0.0.0-20230321000545-74c2419ad056 // indirect
maunium.net/go/maulogger/v2 v2.4.1 // indirect
maunium.net/go/mautrix v0.17.0 // indirect
mvdan.cc/xurls/v2 v2.6.0 // indirect
)
require (
fyne.io/systray v1.12.0 // indirect
github.com/BurntSushi/toml v1.5.0 // indirect
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect
github.com/charmbracelet/lipgloss v1.1.1-0.20250404203927-76690c660834 // indirect
github.com/charmbracelet/x/ansi v0.10.2 // indirect
github.com/charmbracelet/x/cellbuf v0.0.13 // indirect
github.com/charmbracelet/x/exp/slice v0.0.0-20250327172914-2fdc97757edf // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fredbi/uri v1.1.1 // indirect
github.com/fyne-io/gl-js v0.2.0 // indirect
github.com/fyne-io/glfw-js v0.3.0 // indirect
github.com/fyne-io/image v0.1.1 // indirect
github.com/fyne-io/oksvg v0.2.0 // indirect
github.com/go-gl/gl v0.0.0-20231021071112-07e5d0ea2e71 // indirect
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20240506104042-037f3cc74f2a // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-text/render v0.2.0 // indirect
github.com/go-text/typesetting v0.3.3 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/google/jsonschema-go v0.4.2 // indirect
github.com/hack-pad/go-indexeddb v0.3.2 // indirect
github.com/hack-pad/safejs v0.1.0 // indirect
github.com/jeandeaual/go-locale v0.0.0-20250612000132-0ef82f21eade // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/jsummers/gobmp v0.0.0-20230614200233-a9de23ed2e25 // indirect
github.com/libp2p/go-yamux/v5 v5.1.0 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.2.0 // indirect
github.com/moby/patternmatcher v0.6.1 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646 // indirect
github.com/nicksnyder/go-i18n/v2 v2.5.1 // indirect
github.com/otiai10/mint v1.6.3 // indirect
github.com/pion/datachannel v1.6.0 // indirect
github.com/pion/dtls/v3 v3.1.2 // indirect
github.com/pion/ice/v4 v4.2.2 // indirect
github.com/pion/interceptor v0.1.44 // indirect
github.com/pion/logging v0.2.4 // indirect
github.com/pion/mdns/v2 v2.1.0 // indirect
github.com/pion/randutil v0.1.0 // indirect
github.com/pion/rtcp v1.2.16 // indirect
github.com/pion/rtp v1.10.1
github.com/pion/sctp v1.9.4 // indirect
github.com/pion/sdp/v3 v3.0.18 // indirect
github.com/pion/srtp/v3 v3.0.10 // indirect
github.com/pion/stun/v3 v3.1.1 // indirect
github.com/pion/turn/v4 v4.1.4 // indirect
github.com/pion/webrtc/v4 v4.2.11
github.com/prometheus/otlptranslator v1.0.0 // indirect
github.com/rymdport/portal v0.4.2 // indirect
github.com/shirou/gopsutil/v4 v4.26.3 // indirect
github.com/srwiley/oksvg v0.0.0-20221011165216-be6e8873101c // indirect
github.com/srwiley/rasterx v0.0.0-20220730225603-2ab79fcdd4ef // indirect
github.com/wlynxg/anet v0.0.5 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
go.uber.org/mock v0.5.2 // indirect
go.yaml.in/yaml/v2 v2.4.4
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/image v0.38.0 // indirect
golang.org/x/net v0.54.0 // indirect (for websocket)
golang.org/x/oauth2 v0.36.0
golang.org/x/telemetry v0.0.0-20260508192327-42602be52be6 // indirect
golang.org/x/time v0.14.0 // indirect
)
require (
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.11.7 // indirect
github.com/alecthomas/chroma/v2 v2.20.0 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/benbjohnson/clock v1.3.5 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/c-robinson/iplib v1.0.8 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0
github.com/containerd/cgroups v1.1.0 // indirect
github.com/containerd/continuity v0.4.4 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/creachadair/otp v0.5.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/cli v29.4.3+incompatible // indirect
github.com/docker/docker v28.5.2+incompatible
github.com/docker/docker-credential-helpers v0.9.3 // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/flynn/noise v1.1.0 // indirect
github.com/go-audio/audio v1.0.0
github.com/go-audio/riff v1.0.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.22.1 // indirect
github.com/go-openapi/jsonreference v0.21.3 // indirect
github.com/go-openapi/spec v0.22.1 // indirect
github.com/go-openapi/swag v0.25.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/golang/snappy v0.0.5-0.20231225225746-43d5d4cd4e0e // indirect
github.com/google/btree v1.1.3 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/google/pprof v0.0.0-20260402051712-545e8a4df936 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674
github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/henvic/httpretty v0.1.4 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/huin/goupnp v1.3.0 // indirect
github.com/ipfs/boxo v0.39.0 // indirect
github.com/ipfs/go-cid v0.6.1 // indirect
github.com/ipfs/go-datastore v0.9.1 // indirect
github.com/ipfs/go-log/v2 v2.9.2 // indirect
github.com/ipld/go-ipld-prime v0.23.0 // indirect
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
github.com/jaypipes/pcidb v1.1.1 // indirect
github.com/jbenet/go-temp-err-catcher v0.1.0 // indirect
github.com/klauspost/compress v1.18.6
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/koron/go-ssdp v0.0.6 // indirect
github.com/libp2p/go-buffer-pool v0.1.0 // indirect
github.com/libp2p/go-cidranger v1.1.0 // indirect
github.com/libp2p/go-flow-metrics v0.3.0 // indirect
github.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect
github.com/libp2p/go-libp2p-kad-dht v0.40.0 // indirect
github.com/libp2p/go-libp2p-kbucket v0.8.0 // indirect
github.com/libp2p/go-libp2p-pubsub v0.16.0 // indirect
github.com/libp2p/go-libp2p-record v0.3.1 // indirect
github.com/libp2p/go-libp2p-routing-helpers v0.7.5 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/libp2p/go-netroute v0.4.0 // indirect
github.com/libp2p/go-reuseport v0.4.0 // indirect
github.com/libp2p/zeroconf/v2 v2.2.0 // indirect
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.22 // indirect
github.com/mattn/go-runewidth v0.0.17 // indirect
github.com/miekg/dns v1.1.72 // indirect
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/mr-tron/base58 v1.3.0 // indirect
github.com/mudler/go-piper v0.0.0-20241023091659-2494246fd9fc
github.com/mudler/water v0.0.0-20250808092830-dd90dcf09025 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/multiformats/go-base32 v0.1.0 // indirect
github.com/multiformats/go-base36 v0.2.0 // indirect
github.com/multiformats/go-multiaddr v0.16.1
github.com/multiformats/go-multiaddr-dns v0.5.0 // indirect
github.com/multiformats/go-multiaddr-fmt v0.1.0 // indirect
github.com/multiformats/go-multibase v0.3.0 // indirect
github.com/multiformats/go-multicodec v0.10.0 // indirect
github.com/multiformats/go-multihash v0.2.3 // indirect
github.com/multiformats/go-multistream v0.6.1 // indirect
github.com/multiformats/go-varint v0.1.0 // indirect
github.com/nwaples/rardecode v1.1.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pierrec/lz4/v4 v4.1.8 // indirect
github.com/pkg/errors v0.9.1
github.com/pkoukk/tiktoken-go v0.1.7 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polydawn/refmt v0.89.1-0.20231129105047-37766d95467a // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.67.5 // indirect
github.com/prometheus/procfs v0.20.1 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.59.0 // indirect
github.com/quic-go/webtransport-go v0.10.0 // indirect
github.com/rivo/uniseg v0.4.7
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
github.com/smallnest/ringbuffer v0.0.0-20241116012123-461381446e3d // indirect
github.com/songgao/packets v0.0.0-20160404182456-549a10cd4091 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/tklauser/go-sysconf v0.3.16 // indirect
github.com/tklauser/numcpus v0.11.0 // indirect
github.com/ulikunitz/xz v0.5.14 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/vishvananda/netlink v1.3.1 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
github.com/yuin/goldmark v1.7.13 // indirect
github.com/yuin/goldmark-emoji v1.0.6 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.uber.org/dig v1.19.0 // indirect
go.uber.org/fx v1.24.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.28.0 // indirect
golang.org/x/crypto v0.51.0
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/mod v0.36.0 // indirect
golang.org/x/sync v0.20.0
golang.org/x/sys v0.45.0 // indirect
golang.org/x/term v0.43.0
golang.org/x/text v0.37.0
golang.org/x/tools v0.45.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
golang.zx2c4.com/wireguard v0.0.0-20250521234502-f333402bd9cb // indirect
golang.zx2c4.com/wireguard/windows v0.6.1 // indirect
gonum.org/v1/gonum v0.17.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57 // indirect
gopkg.in/fsnotify.v1 v1.4.7 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
howett.net/plist v1.0.2-0.20250314012144-ee69052608d9 // indirect
lukechampine.com/blake3 v1.4.1 // indirect
)