Compare commits

...

17 Commits

Author SHA1 Message Date
dependabot[bot]
fd1cad3e9b chore(deps): bump github.com/swaggo/echo-swagger from 1.4.1 to 1.5.0
Bumps [github.com/swaggo/echo-swagger](https://github.com/swaggo/echo-swagger) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/swaggo/echo-swagger/releases)
- [Commits](https://github.com/swaggo/echo-swagger/compare/v1.4.1...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/swaggo/echo-swagger
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-24 01:31:24 +00:00
dependabot[bot]
d19dcac863 chore(deps): bump github.com/mudler/cogito from 0.9.1-0.20260217143801-bb7f986ed2c7 to 0.9.1 (#8632)
chore(deps): bump github.com/mudler/cogito

Bumps [github.com/mudler/cogito](https://github.com/mudler/cogito) from 0.9.1-0.20260217143801-bb7f986ed2c7 to 0.9.1.
- [Release notes](https://github.com/mudler/cogito/releases)
- [Commits](https://github.com/mudler/cogito/commits/v0.9.1)

---
updated-dependencies:
- dependency-name: github.com/mudler/cogito
  dependency-version: 0.9.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-24 01:29:47 +00:00
dependabot[bot]
fd42675bec chore(deps): bump goreleaser/goreleaser-action from 6 to 7 (#8634)
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 6 to 7.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 23:27:49 +01:00
dependabot[bot]
3391538806 chore(deps): bump actions/stale from 10.1.1 to 10.2.0 (#8633)
Bumps [actions/stale](https://github.com/actions/stale) from 10.1.1 to 10.2.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](997185467f...b5d41d4e1d)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 10.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 23:27:20 +01:00
dependabot[bot]
c4f879c4ea chore(deps): bump github.com/gpustack/gguf-parser-go from 0.23.1 to 0.24.0 (#8631)
chore(deps): bump github.com/gpustack/gguf-parser-go

Bumps [github.com/gpustack/gguf-parser-go](https://github.com/gpustack/gguf-parser-go) from 0.23.1 to 0.24.0.
- [Release notes](https://github.com/gpustack/gguf-parser-go/releases)
- [Commits](https://github.com/gpustack/gguf-parser-go/compare/v0.23.1...v0.24.0)

---
updated-dependencies:
- dependency-name: github.com/gpustack/gguf-parser-go
  dependency-version: 0.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 23:26:41 +01:00
dependabot[bot]
b7e0de54fe chore(deps): bump github.com/anthropics/anthropic-sdk-go from 1.22.0 to 1.26.0 (#8630)
chore(deps): bump github.com/anthropics/anthropic-sdk-go

Bumps [github.com/anthropics/anthropic-sdk-go](https://github.com/anthropics/anthropic-sdk-go) from 1.22.0 to 1.26.0.
- [Release notes](https://github.com/anthropics/anthropic-sdk-go/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-go/compare/v1.22.0...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/anthropics/anthropic-sdk-go
  dependency-version: 1.26.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 23:26:15 +01:00
dependabot[bot]
f0868acdf3 chore(deps): bump fyne.io/fyne/v2 from 2.7.2 to 2.7.3 (#8629)
Bumps [fyne.io/fyne/v2](https://github.com/fyne-io/fyne) from 2.7.2 to 2.7.3.
- [Release notes](https://github.com/fyne-io/fyne/releases)
- [Changelog](https://github.com/fyne-io/fyne/blob/v2.7.3/CHANGELOG.md)
- [Commits](https://github.com/fyne-io/fyne/compare/v2.7.2...v2.7.3)

---
updated-dependencies:
- dependency-name: fyne.io/fyne/v2
  dependency-version: 2.7.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 23:25:46 +01:00
LocalAI [bot]
9a5b5ee8a9 chore: ⬆️ Update ggml-org/llama.cpp to b68a83e641b3ebe6465970b34e99f3f0e0a0b21a (#8628)
⬆️ Update ggml-org/llama.cpp

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2026-02-23 22:02:40 +00:00
Lukas Schaefer
ed0bfb8732 fix: rename json_verbose to verbose_json (#8627)
Signed-off-by: Lukas Schaefer <lukas@lschaefer.xyz>
2026-02-23 17:57:06 +00:00
Richard Palethorpe
be84b1d258 feat(traces): Use accordian instead of pop-ups (#8626)
Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-02-23 13:07:41 +01:00
Andres
cbedcc9091 fix(api): Downgrade health/readiness check to debug (#8625)
Downgrade health/readiness check to debug

Signed-off-by: Andres Smith <andressmithdev@pm.me>
2026-02-23 11:58:04 +01:00
Andres
e45d63c86e fix(cli): Fix watchdog running constantly and spamming logs (#8624)
* Fix watchdog running constantly and spamming logs

Signed-off-by: Andres Smith <andressmithdev@pm.me>

* Update docs

Signed-off-by: Andres Smith <andressmithdev@pm.me>

---------

Signed-off-by: Andres Smith <andressmithdev@pm.me>
2026-02-23 11:57:28 +01:00
LocalAI [bot]
f40c8dd0ce chore: ⬆️ Update ggml-org/llama.cpp to 2b6dfe824de8600c061ef91ce5cc5c307f97112c (#8622)
⬆️ Update ggml-org/llama.cpp

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2026-02-23 09:30:58 +00:00
LocalAI [bot]
559ab99890 docs: update diffusers multi-GPU documentation to mention tensor_parallel_size configuration (#8621)
* docs: update diffusers multi-GPU documentation to mention tensor_parallel_size configuration

* chore: revert backend/python/diffusers/README.md to original content

---------

Co-authored-by: Your Name <you@example.com>
2026-02-22 18:17:23 +01:00
LocalAI [bot]
91f2dd5820 chore: ⬆️ Update ggml-org/llama.cpp to f75c4e8bf52ea480ece07fd3d9a292f1d7f04bc5 (#8619)
⬆️ Update ggml-org/llama.cpp

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2026-02-22 13:20:08 +01:00
LocalAI [bot]
8250815763 docs: ⬆️ update docs version mudler/LocalAI (#8618)
⬆️ Update docs version mudler/LocalAI

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2026-02-21 21:18:40 +00:00
Richard Palethorpe
b1b67b973e fix(realtime): Add functions to conversation history (#8616)
Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-02-21 19:03:49 +01:00
17 changed files with 277 additions and 285 deletions

View File

@@ -18,7 +18,7 @@ jobs:
with:
go-version: 1.23
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
uses: goreleaser/goreleaser-action@v7
with:
version: v2.11.0
args: release --clean

View File

@@ -11,7 +11,7 @@ jobs:
if: github.repository == 'mudler/LocalAI'
runs-on: ubuntu-latest
steps:
- uses: actions/stale@997185467fa4f803885201cee163a9f38240193d # v9
- uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v9
with:
stale-issue-message: 'This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.'
stale-pr-message: 'This PR is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 10 days.'

1
CLAUDE.md Symbolic link
View File

@@ -0,0 +1 @@
AGENTS.md

View File

@@ -1,5 +1,5 @@
LLAMA_VERSION?=ba3b9c8844aca35ecb40d31886686326f22d2214
LLAMA_VERSION?=b68a83e641b3ebe6465970b34e99f3f0e0a0b21a
LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
CMAKE_ARGS?=

View File

@@ -71,6 +71,7 @@ type RunCMD struct {
WatchdogIdleTimeout string `env:"LOCALAI_WATCHDOG_IDLE_TIMEOUT,WATCHDOG_IDLE_TIMEOUT" default:"15m" help:"Threshold beyond which an idle backend should be stopped" group:"backends"`
EnableWatchdogBusy bool `env:"LOCALAI_WATCHDOG_BUSY,WATCHDOG_BUSY" default:"false" help:"Enable watchdog for stopping backends that are busy longer than the watchdog-busy-timeout" group:"backends"`
WatchdogBusyTimeout string `env:"LOCALAI_WATCHDOG_BUSY_TIMEOUT,WATCHDOG_BUSY_TIMEOUT" default:"5m" help:"Threshold beyond which a busy backend should be stopped" group:"backends"`
WatchdogInterval string `env:"LOCALAI_WATCHDOG_INTERVAL,WATCHDOG_INTERVAL" default:"500ms" help:"Interval between watchdog checks (e.g., 500ms, 5s, 1m) (default: 500ms)" group:"backends"`
EnableMemoryReclaimer bool `env:"LOCALAI_MEMORY_RECLAIMER,MEMORY_RECLAIMER,LOCALAI_GPU_RECLAIMER,GPU_RECLAIMER" default:"false" help:"Enable memory threshold monitoring to auto-evict backends when memory usage exceeds threshold (uses GPU VRAM if available, otherwise RAM)" group:"backends"`
MemoryReclaimerThreshold float64 `env:"LOCALAI_MEMORY_RECLAIMER_THRESHOLD,MEMORY_RECLAIMER_THRESHOLD,LOCALAI_GPU_RECLAIMER_THRESHOLD,GPU_RECLAIMER_THRESHOLD" default:"0.95" help:"Memory usage threshold (0.0-1.0) that triggers backend eviction (default 0.95 = 95%%)" group:"backends"`
ForceEvictionWhenBusy bool `env:"LOCALAI_FORCE_EVICTION_WHEN_BUSY,FORCE_EVICTION_WHEN_BUSY" default:"false" help:"Force eviction even when models have active API calls (default: false for safety)" group:"backends"`
@@ -215,6 +216,13 @@ func (r *RunCMD) Run(ctx *cliContext.Context) error {
}
opts = append(opts, config.SetWatchDogBusyTimeout(dur))
}
if r.WatchdogInterval != "" {
dur, err := time.ParseDuration(r.WatchdogInterval)
if err != nil {
return err
}
opts = append(opts, config.SetWatchDogInterval(dur))
}
}
// Handle memory reclaimer (uses GPU VRAM if available, otherwise RAM)

View File

@@ -31,8 +31,8 @@ type TranscriptCMD struct {
ModelsPath string `env:"LOCALAI_MODELS_PATH,MODELS_PATH" type:"path" default:"${basepath}/models" help:"Path containing models used for inferencing" group:"storage"`
BackendGalleries string `env:"LOCALAI_BACKEND_GALLERIES,BACKEND_GALLERIES" help:"JSON list of backend galleries" group:"backends" default:"${backends}"`
Prompt string `short:"p" help:"Previous transcribed text or words that hint at what the model should expect"`
ResponseFormat schema.TranscriptionResponseFormatType `short:"f" default:"" help:"Response format for Whisper models, can be one of (txt, lrc, srt, vtt, json, json_verbose)"`
PrettyPrint bool `help:"Used with response_format json or json_verbose for pretty printing"`
ResponseFormat schema.TranscriptionResponseFormatType `short:"f" default:"" help:"Response format for Whisper models, can be one of (txt, lrc, srt, vtt, json, verbose_json)"`
PrettyPrint bool `help:"Used with response_format json or verbose_json for pretty printing"`
}
func (t *TranscriptCMD) Run(ctx *cliContext.Context) error {

View File

@@ -98,10 +98,11 @@ func NewApplicationConfig(o ...AppOption) *ApplicationConfig {
Context: context.Background(),
UploadLimitMB: 15,
Debug: true,
AgentJobRetentionDays: 30, // Default: 30 days
LRUEvictionMaxRetries: 30, // Default: 30 retries
LRUEvictionRetryInterval: 1 * time.Second, // Default: 1 second
TracingMaxItems: 1024,
AgentJobRetentionDays: 30, // Default: 30 days
LRUEvictionMaxRetries: 30, // Default: 30 retries
LRUEvictionRetryInterval: 1 * time.Second, // Default: 1 second
WatchDogInterval: 500 * time.Millisecond, // Default: 500ms
TracingMaxItems: 1024,
PathWithoutAuth: []string{
"/static/",
"/generated-audio/",
@@ -208,6 +209,12 @@ func SetWatchDogIdleTimeout(t time.Duration) AppOption {
}
}
func SetWatchDogInterval(t time.Duration) AppOption {
return func(o *ApplicationConfig) {
o.WatchDogInterval = t
}
}
// EnableMemoryReclaimer enables memory threshold monitoring.
// When enabled, the watchdog will evict backends if memory usage exceeds the threshold.
// Works with GPU VRAM if available, otherwise uses system RAM.
@@ -642,7 +649,7 @@ func (o *ApplicationConfig) ToRuntimeSettings() RuntimeSettings {
AutoloadBackendGalleries: &autoloadBackendGalleries,
ApiKeys: &apiKeys,
AgentJobRetentionDays: &agentJobRetentionDays,
OpenResponsesStoreTTL: &openResponsesStoreTTL,
OpenResponsesStoreTTL: &openResponsesStoreTTL,
}
}

View File

@@ -29,6 +29,8 @@ import (
//go:embed static/*
var embedDirStatic embed.FS
var quietPaths = []string{"/api/operations", "/api/resources", "/healthz", "/readyz"}
// @title LocalAI API
// @version 2.0.0
// @description The LocalAI Rest API.
@@ -109,10 +111,17 @@ func API(application *application.Application) (*echo.Echo, error) {
res := c.Response()
err := next(c)
// Fix for #7989: Reduce log verbosity of Web UI polling and resources API
// If the path is /api/operations or /api/resources and the request was successful (200),
// we log it at DEBUG level (hidden by default) instead of INFO.
if (req.URL.Path == "/api/operations" || req.URL.Path == "/api/resources") && res.Status == 200 {
// Fix for #7989: Reduce log verbosity of Web UI polling, resources API, and health checks
// These paths are logged at DEBUG level (hidden by default) instead of INFO.
isQuietPath := false
for _, path := range quietPaths {
if req.URL.Path == path {
isQuietPath = true
break
}
}
if isQuietPath && res.Status == 200 {
xlog.Debug("HTTP request", "method", req.Method, "path", req.URL.Path, "status", res.Status)
} else {
xlog.Info("HTTP request", "method", req.Method, "path", req.URL.Path, "status", res.Status)

View File

@@ -1046,6 +1046,27 @@ func triggerResponse(session *Session, conv *Conversation, c *LockedWebsocket, o
Content: content.Text,
})
}
} else if item.FunctionCall != nil {
conversationHistory = append(conversationHistory, schema.Message{
Role: string(types.MessageRoleAssistant),
ToolCalls: []schema.ToolCall{
{
ID: item.FunctionCall.CallID,
Type: "function",
FunctionCall: schema.FunctionCall{
Name: item.FunctionCall.Name,
Arguments: item.FunctionCall.Arguments,
},
},
},
})
} else if item.FunctionCallOutput != nil {
conversationHistory = append(conversationHistory, schema.Message{
Role: "tool",
Name: item.FunctionCallOutput.CallID,
Content: item.FunctionCallOutput.Output,
StringContent: item.FunctionCallOutput.Output,
})
}
}
conv.Lock.Unlock()

View File

@@ -135,26 +135,44 @@
<table class="w-full border-collapse">
<thead>
<tr class="border-b border-[var(--color-bg-secondary)]">
<th class="w-8 p-2"></th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Method</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Path</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Status</th>
<th class="text-right p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Actions</th>
</tr>
</thead>
<tbody>
<template x-for="(trace, index) in traces" :key="index">
<tr class="hover:bg-[var(--color-bg-secondary)]/50 border-b border-[var(--color-bg-secondary)] transition-colors">
<template x-for="(trace, index) in traces" :key="index">
<tbody>
<tr @click="toggleTrace(index)"
class="cursor-pointer hover:bg-[var(--color-bg-secondary)]/50 border-b border-[var(--color-bg-secondary)] transition-colors">
<td class="p-2 w-8 text-center">
<i class="fas fa-chevron-right text-xs text-[var(--color-text-secondary)] transition-transform duration-200"
:class="expandedTraces[index] ? 'rotate-90' : ''"></i>
</td>
<td class="p-2" x-text="trace.request.method"></td>
<td class="p-2" x-text="trace.request.path"></td>
<td class="p-2" x-text="trace.response.status"></td>
<td class="p-2 text-right">
<button @click="showDetails(index)" class="text-[var(--color-primary)]/60 hover:text-[var(--color-primary)] hover:bg-[var(--color-primary)]/10 rounded p-1 transition-colors">
<i class="fas fa-eye text-xs"></i>
</button>
</tr>
<tr x-show="expandedTraces[index]">
<td colspan="4" class="p-0">
<div class="p-4 bg-[var(--color-bg-secondary)]/30 border-b border-[var(--color-bg-secondary)]">
<div class="grid grid-cols-1 lg:grid-cols-2 gap-4">
<div>
<h4 class="text-sm font-semibold text-[var(--color-text-primary)] mb-2">Request Body</h4>
<pre class="overflow-auto max-h-[70vh] p-3 rounded-lg bg-[var(--color-bg-primary)] border border-[var(--color-border-subtle)] text-xs font-mono text-[var(--color-text-secondary)] whitespace-pre-wrap break-words"
x-text="formatTraceBody(trace.request.body)"></pre>
</div>
<div>
<h4 class="text-sm font-semibold text-[var(--color-text-primary)] mb-2">Response Body</h4>
<pre class="overflow-auto max-h-[70vh] p-3 rounded-lg bg-[var(--color-bg-primary)] border border-[var(--color-border-subtle)] text-xs font-mono text-[var(--color-text-secondary)] whitespace-pre-wrap break-words"
x-text="formatTraceBody(trace.response.body)"></pre>
</div>
</div>
</div>
</td>
</tr>
</template>
</tbody>
</tbody>
</template>
</table>
<div x-show="traces.length === 0" class="text-center py-8 text-[var(--color-text-secondary)] text-sm">
No API traces recorded yet.
@@ -168,18 +186,23 @@
<table class="w-full border-collapse">
<thead>
<tr class="border-b border-[var(--color-bg-secondary)]">
<th class="w-8 p-2"></th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Type</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Timestamp</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Model</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Summary</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Duration</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Status</th>
<th class="text-right p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Actions</th>
</tr>
</thead>
<tbody>
<template x-for="(trace, index) in backendTraces" :key="index">
<tr class="hover:bg-[var(--color-bg-secondary)]/50 border-b border-[var(--color-bg-secondary)] transition-colors">
<template x-for="(trace, index) in backendTraces" :key="index">
<tbody>
<tr @click="toggleBackendTrace(index)"
class="cursor-pointer hover:bg-[var(--color-bg-secondary)]/50 border-b border-[var(--color-bg-secondary)] transition-colors">
<td class="p-2 w-8 text-center">
<i class="fas fa-chevron-right text-xs text-[var(--color-text-secondary)] transition-transform duration-200"
:class="expandedBackendTraces[index] ? 'rotate-90' : ''"></i>
</td>
<td class="p-2">
<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium"
:class="getTypeClass(trace.type)"
@@ -197,14 +220,82 @@
<i class="fas fa-times-circle text-red-500 text-xs" :title="trace.error"></i>
</template>
</td>
<td class="p-2 text-right">
<button @click="showBackendDetails(index)" class="text-[var(--color-primary)]/60 hover:text-[var(--color-primary)] hover:bg-[var(--color-primary)]/10 rounded p-1 transition-colors">
<i class="fas fa-eye text-xs"></i>
</button>
</tr>
<tr x-show="expandedBackendTraces[index]">
<td colspan="7" class="p-0">
<div class="p-4 bg-[var(--color-bg-secondary)]/30 border-b border-[var(--color-bg-secondary)]">
<!-- Header info -->
<div class="grid grid-cols-2 md:grid-cols-4 gap-3 mb-4">
<div class="bg-[var(--color-bg-primary)] rounded-lg p-3 border border-[var(--color-border-subtle)]">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Type</div>
<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium"
:class="getTypeClass(trace.type)"
x-text="trace.type"></span>
</div>
<div class="bg-[var(--color-bg-primary)] rounded-lg p-3 border border-[var(--color-border-subtle)]">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Model</div>
<div class="text-sm font-medium" x-text="trace.model_name || '-'"></div>
</div>
<div class="bg-[var(--color-bg-primary)] rounded-lg p-3 border border-[var(--color-border-subtle)]">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Backend</div>
<div class="text-sm font-medium" x-text="trace.backend || '-'"></div>
</div>
<div class="bg-[var(--color-bg-primary)] rounded-lg p-3 border border-[var(--color-border-subtle)]">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Duration</div>
<div class="text-sm font-medium" x-text="formatDuration(trace.duration)"></div>
</div>
</div>
<!-- Error banner -->
<div x-show="trace.error" class="bg-red-500/10 border border-red-500/30 rounded-lg p-3 mb-4">
<div class="flex items-center gap-2">
<i class="fas fa-exclamation-triangle text-red-500 text-sm"></i>
<span class="text-sm text-red-400" x-text="trace.error"></span>
</div>
</div>
<!-- Data fields as nested accordions -->
<template x-if="trace.data && Object.keys(trace.data).length > 0">
<div>
<h4 class="text-sm font-semibold text-[var(--color-text-primary)] mb-2">Data Fields</h4>
<div class="border border-[var(--color-border-subtle)] rounded-lg overflow-hidden">
<template x-for="[key, value] in Object.entries(trace.data)" :key="key">
<div class="border-b border-[var(--color-border-subtle)] last:border-b-0">
<!-- Field header row -->
<div @click="isLargeValue(value) && toggleBackendField(index, key)"
class="flex items-center gap-2 px-3 py-2 hover:bg-[var(--color-bg-primary)]/50 transition-colors"
:class="isLargeValue(value) ? 'cursor-pointer' : ''">
<template x-if="isLargeValue(value)">
<i class="fas fa-chevron-right text-[10px] text-[var(--color-text-secondary)] transition-transform duration-200 w-3 flex-shrink-0"
:class="isBackendFieldExpanded(index, key) ? 'rotate-90' : ''"></i>
</template>
<template x-if="!isLargeValue(value)">
<span class="w-3 flex-shrink-0"></span>
</template>
<span class="text-sm font-mono text-[var(--color-primary)] flex-shrink-0" x-text="key"></span>
<template x-if="!isLargeValue(value)">
<span class="font-mono text-xs text-[var(--color-text-secondary)]" x-text="formatValue(value)"></span>
</template>
<template x-if="isLargeValue(value) && !isBackendFieldExpanded(index, key)">
<span class="text-xs text-[var(--color-text-secondary)] truncate" x-text="truncateValue(value, 120)"></span>
</template>
</div>
<!-- Expanded field value -->
<div x-show="isLargeValue(value) && isBackendFieldExpanded(index, key)"
class="px-3 pb-3">
<pre class="overflow-auto max-h-[70vh] p-3 rounded-lg bg-[var(--color-bg-primary)] border border-[var(--color-border-subtle)] text-xs font-mono text-[var(--color-text-secondary)] whitespace-pre-wrap break-words"
x-text="formatLargeValue(value)"></pre>
</div>
</div>
</template>
</div>
</div>
</template>
</div>
</td>
</tr>
</template>
</tbody>
</tbody>
</template>
</table>
<div x-show="backendTraces.length === 0" class="text-center py-8 text-[var(--color-text-secondary)] text-sm">
No backend traces recorded yet.
@@ -212,149 +303,20 @@
</div>
</div>
<!-- API Trace Details Modal -->
<div x-show="selectedTrace !== null" class="fixed inset-0 bg-black/50 flex items-center justify-center z-50" @click="selectedTrace = null">
<div class="bg-[var(--color-bg-secondary)] rounded-lg p-6 max-w-4xl w-full max-h-[90vh] overflow-auto" @click.stop>
<div class="flex justify-between mb-4">
<h2 class="h3">API Trace Details</h2>
<button @click="selectedTrace = null" class="text-[var(--color-text-secondary)] hover:text-[var(--color-text-primary)]">
<i class="fas fa-times"></i>
</button>
</div>
<div class="grid grid-cols-2 gap-4">
<div>
<h3 class="text-lg font-semibold mb-2">Request Body</h3>
<div id="requestEditor" class="h-96 border border-[var(--color-primary-border)]/20"></div>
</div>
<div>
<h3 class="text-lg font-semibold mb-2">Response Body</h3>
<div id="responseEditor" class="h-96 border border-[var(--color-primary-border)]/20"></div>
</div>
</div>
</div>
</div>
<!-- Backend Trace Details Modal -->
<div x-show="selectedBackendTrace !== null" class="fixed inset-0 bg-black/50 flex items-center justify-center z-50" @click="selectedBackendTrace = null; detailKey = null; detailValue = null;">
<div class="bg-[var(--color-bg-secondary)] rounded-lg p-6 max-w-4xl w-full max-h-[90vh] overflow-auto" @click.stop>
<template x-if="selectedBackendTrace !== null">
<div>
<div class="flex justify-between mb-4">
<h2 class="h3">Backend Trace Details</h2>
<button @click="selectedBackendTrace = null; detailKey = null; detailValue = null;" class="text-[var(--color-text-secondary)] hover:text-[var(--color-text-primary)]">
<i class="fas fa-times"></i>
</button>
</div>
<!-- Header info -->
<div class="grid grid-cols-4 gap-4 mb-4">
<div class="bg-[var(--color-bg-primary)] rounded p-3">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Type</div>
<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium"
:class="getTypeClass(backendTraces[selectedBackendTrace].type)"
x-text="backendTraces[selectedBackendTrace].type"></span>
</div>
<div class="bg-[var(--color-bg-primary)] rounded p-3">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Model</div>
<div class="text-sm font-medium" x-text="backendTraces[selectedBackendTrace].model_name || '-'"></div>
</div>
<div class="bg-[var(--color-bg-primary)] rounded p-3">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Backend</div>
<div class="text-sm font-medium" x-text="backendTraces[selectedBackendTrace].backend || '-'"></div>
</div>
<div class="bg-[var(--color-bg-primary)] rounded p-3">
<div class="text-xs text-[var(--color-text-secondary)] mb-1">Duration</div>
<div class="text-sm font-medium" x-text="formatDuration(backendTraces[selectedBackendTrace].duration)"></div>
</div>
</div>
<!-- Error banner -->
<div x-show="backendTraces[selectedBackendTrace].error" class="bg-red-500/10 border border-red-500/30 rounded-lg p-3 mb-4">
<div class="flex items-center gap-2">
<i class="fas fa-exclamation-triangle text-red-500 text-sm"></i>
<span class="text-sm text-red-400" x-text="backendTraces[selectedBackendTrace].error"></span>
</div>
</div>
<!-- Data fields table -->
<div class="overflow-x-auto">
<table class="w-full border-collapse">
<thead>
<tr class="border-b border-[var(--color-bg-primary)]">
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)] w-1/4">Field</th>
<th class="text-left p-2 text-xs font-semibold text-[var(--color-text-secondary)]">Value</th>
</tr>
</thead>
<tbody>
<template x-for="[key, value] in getDataEntries(selectedBackendTrace)" :key="key">
<tr class="border-b border-[var(--color-bg-primary)] hover:bg-[var(--color-bg-primary)]/50 transition-colors">
<td class="p-2 text-sm font-mono text-[var(--color-primary)]" x-text="key"></td>
<td class="p-2 text-sm">
<template x-if="isLargeValue(value)">
<button @click="showValueDetail(key, value)"
class="text-left max-w-full">
<span class="block truncate max-w-lg text-[var(--color-text-secondary)]" x-text="truncateValue(value, 120)"></span>
<span class="text-xs text-[var(--color-primary)] hover:underline mt-0.5 inline-block">View full value</span>
</button>
</template>
<template x-if="!isLargeValue(value)">
<span class="font-mono text-xs" x-text="formatValue(value)"></span>
</template>
</td>
</tr>
</template>
</tbody>
</table>
</div>
</div>
</template>
</div>
</div>
<!-- Value Detail Modal -->
<div x-show="detailValue !== null" class="fixed inset-0 bg-black/50 flex items-center justify-center z-[60]" @click="detailValue = null; detailKey = null;">
<div class="bg-[var(--color-bg-secondary)] rounded-lg p-6 max-w-4xl w-full max-h-[90vh] overflow-auto" @click.stop>
<div class="flex justify-between mb-4">
<h2 class="h3 font-mono" x-text="detailKey"></h2>
<button @click="detailValue = null; detailKey = null;" class="text-[var(--color-text-secondary)] hover:text-[var(--color-text-primary)]">
<i class="fas fa-times"></i>
</button>
</div>
<div id="detailEditor" class="h-[70vh] border border-[var(--color-primary-border)]/20"></div>
</div>
</div>
</div>
</div>
<!-- CodeMirror -->
<link rel="stylesheet" href="static/assets/codemirror.min.css">
<script src="static/assets/codemirror.min.js"></script>
<script src="static/assets/javascript.min.js"></script>
<!-- Styles from model-editor -->
<style>
.CodeMirror {
height: 100% !important;
font-family: monospace;
}
</style>
<script>
function tracesApp() {
return {
activeTab: 'api',
traces: [],
backendTraces: [],
selectedTrace: null,
selectedBackendTrace: null,
detailKey: null,
detailValue: null,
requestEditor: null,
responseEditor: null,
detailEditor: null,
expandedTraces: {},
expandedBackendTraces: {},
expandedBackendFields: {},
notifications: [],
settings: {
enable_tracing: false,
@@ -474,6 +436,7 @@ function tracesApp() {
if (confirm('Clear all API traces?')) {
await fetch('/api/traces/clear', { method: 'POST' });
this.traces = [];
this.expandedTraces = {};
}
},
@@ -481,101 +444,67 @@ function tracesApp() {
if (confirm('Clear all backend traces?')) {
await fetch('/api/backend-traces/clear', { method: 'POST' });
this.backendTraces = [];
this.expandedBackendTraces = {};
this.expandedBackendFields = {};
}
},
showDetails(index) {
this.selectedTrace = index;
this.$nextTick(() => {
const trace = this.traces[index];
const decodeBase64 = (base64) => {
const binaryString = atob(base64);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return new TextDecoder().decode(bytes);
};
const formatBody = (bodyText) => {
try {
const json = JSON.parse(bodyText);
return JSON.stringify(json, null, 2);
} catch {
return bodyText;
}
};
const reqBody = formatBody(decodeBase64(trace.request.body));
const resBody = formatBody(decodeBase64(trace.response.body));
if (!this.requestEditor) {
this.requestEditor = CodeMirror(document.getElementById('requestEditor'), {
value: reqBody,
mode: 'javascript',
json: true,
theme: 'default',
lineNumbers: true,
readOnly: true,
lineWrapping: true
});
} else {
this.requestEditor.setValue(reqBody);
}
if (!this.responseEditor) {
this.responseEditor = CodeMirror(document.getElementById('responseEditor'), {
value: resBody,
mode: 'javascript',
json: true,
theme: 'default',
lineNumbers: true,
readOnly: true,
lineWrapping: true
});
} else {
this.responseEditor.setValue(resBody);
}
});
toggleTrace(index) {
this.expandedTraces = {
...this.expandedTraces,
[index]: !this.expandedTraces[index]
};
},
showBackendDetails(index) {
this.selectedBackendTrace = index;
toggleBackendTrace(index) {
this.expandedBackendTraces = {
...this.expandedBackendTraces,
[index]: !this.expandedBackendTraces[index]
};
},
showValueDetail(key, value) {
this.detailKey = key;
let formatted = '';
toggleBackendField(index, key) {
const fieldKey = index + '-' + key;
this.expandedBackendFields = {
...this.expandedBackendFields,
[fieldKey]: !this.expandedBackendFields[fieldKey]
};
},
isBackendFieldExpanded(index, key) {
return !!this.expandedBackendFields[index + '-' + key];
},
formatTraceBody(body) {
try {
const binaryString = atob(body);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
const text = new TextDecoder().decode(bytes);
try {
return JSON.stringify(JSON.parse(text), null, 2);
} catch {
return text;
}
} catch {
return body || '';
}
},
formatLargeValue(value) {
if (typeof value === 'string') {
try {
const parsed = JSON.parse(value);
formatted = JSON.stringify(parsed, null, 2);
return JSON.stringify(JSON.parse(value), null, 2);
} catch {
formatted = value;
return value;
}
} else if (typeof value === 'object') {
formatted = JSON.stringify(value, null, 2);
} else {
formatted = String(value);
}
this.detailValue = formatted;
this.$nextTick(() => {
const el = document.getElementById('detailEditor');
if (el) {
el.innerHTML = '';
this.detailEditor = CodeMirror(el, {
value: formatted,
mode: 'javascript',
json: true,
theme: 'default',
lineNumbers: true,
readOnly: true,
lineWrapping: true
});
}
});
if (typeof value === 'object') {
return JSON.stringify(value, null, 2);
}
return String(value);
},
formatTimestamp(ts) {
@@ -623,12 +552,6 @@ function tracesApp() {
if (typeof value === 'boolean') return value ? 'true' : 'false';
if (typeof value === 'object') return JSON.stringify(value);
return String(value);
},
getDataEntries(index) {
const trace = this.backendTraces[index];
if (!trace || !trace.data) return [];
return Object.entries(trace.data);
}
}
}

View File

@@ -115,7 +115,7 @@ const (
TranscriptionResponseFormatVtt = TranscriptionResponseFormatType("vtt")
TranscriptionResponseFormatLrc = TranscriptionResponseFormatType("lrc")
TranscriptionResponseFormatJson = TranscriptionResponseFormatType("json")
TranscriptionResponseFormatJsonVerbose = TranscriptionResponseFormatType("json_verbose")
TranscriptionResponseFormatJsonVerbose = TranscriptionResponseFormatType("verbose_json")
)
type ChatCompletionResponseFormat struct {

View File

@@ -60,6 +60,22 @@ diffusers:
scheduler_type: "k_dpmpp_sde"
```
### Multi-GPU Support
For multi-GPU support with diffusers, you need to configure the model with `tensor_parallel_size` set to the number of GPUs you want to use.
```yaml
name: stable-diffusion-multigpu
model: stabilityai/stable-diffusion-xl-base-1.0
backend: diffusers
parameters:
tensor_parallel_size: 2 # Number of GPUs to use
```
The `tensor_parallel_size` parameter is set in the gRPC proto configuration (in `ModelOptions` message, field 55). When this is set to a value greater than 1, the diffusers backend automatically enables `device_map="auto"` to distribute the model across multiple GPUs.
When using diffusers with multiple GPUs, ensure you have sufficient GPU memory across all devices. The model will be automatically distributed across available GPUs. For optimal performance, use GPUs of the same type and memory capacity.
## CUDA(NVIDIA) acceleration
### Requirements

View File

@@ -57,7 +57,7 @@ Result:
---
You can also specify the `response_format` parameter to be one of `lrc`, `srt`, `vtt`, `text`, `json` or `json_verbose` (default):
You can also specify the `response_format` parameter to be one of `lrc`, `srt`, `vtt`, `text`, `json` or `verbose_json` (default):
```bash
## Send the example audio file to the transcriptions endpoint
curl http://localhost:8080/v1/audio/transcriptions -H "Content-Type: multipart/form-data" -F file="@$PWD/gb1.ogg" -F model="whisper-1" -F response_format="srt"

View File

@@ -46,6 +46,7 @@ Complete reference for all LocalAI command-line interface (CLI) parameters and e
| `--watchdog-idle-timeout` | `15m` | Threshold beyond which an idle backend should be stopped | `$LOCALAI_WATCHDOG_IDLE_TIMEOUT`, `$WATCHDOG_IDLE_TIMEOUT` |
| `--enable-watchdog-busy` | `false` | Enable watchdog for stopping backends that are busy longer than the watchdog-busy-timeout | `$LOCALAI_WATCHDOG_BUSY`, `$WATCHDOG_BUSY` |
| `--watchdog-busy-timeout` | `5m` | Threshold beyond which a busy backend should be stopped | `$LOCALAI_WATCHDOG_BUSY_TIMEOUT`, `$WATCHDOG_BUSY_TIMEOUT` |
| `--watchdog-interval` | `500ms` | Interval between watchdog checks (e.g., `500ms`, `5s`, `1m`) | `$LOCALAI_WATCHDOG_INTERVAL`, `$WATCHDOG_INTERVAL` |
| `--force-eviction-when-busy` | `false` | Force eviction even when models have active API calls (default: false for safety). **Warning:** Enabling this can interrupt active requests | `$LOCALAI_FORCE_EVICTION_WHEN_BUSY`, `$FORCE_EVICTION_WHEN_BUSY` |
| `--lru-eviction-max-retries` | `30` | Maximum number of retries when waiting for busy models to become idle before eviction | `$LOCALAI_LRU_EVICTION_MAX_RETRIES`, `$LRU_EVICTION_MAX_RETRIES` |
| `--lru-eviction-retry-interval` | `1s` | Interval between retries when waiting for busy models to become idle (e.g., `1s`, `2s`) | `$LOCALAI_LRU_EVICTION_RETRY_INTERVAL`, `$LRU_EVICTION_RETRY_INTERVAL` |

View File

@@ -1,3 +1,3 @@
{
"version": "v3.12.0"
"version": "v3.12.1"
}

21
go.mod
View File

@@ -1,15 +1,13 @@
module github.com/mudler/LocalAI
go 1.24.4
toolchain go1.24.5
go 1.25.0
require (
dario.cat/mergo v1.0.2
fyne.io/fyne/v2 v2.7.2
fyne.io/fyne/v2 v2.7.3
github.com/Masterminds/sprig/v3 v3.3.0
github.com/alecthomas/kong v1.14.0
github.com/anthropics/anthropic-sdk-go v1.22.0
github.com/anthropics/anthropic-sdk-go v1.26.0
github.com/charmbracelet/glamour v0.10.0
github.com/containerd/containerd v1.7.30
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8
@@ -21,7 +19,7 @@ require (
github.com/gofrs/flock v0.13.0
github.com/google/go-containerregistry v0.20.7
github.com/google/uuid v1.6.0
github.com/gpustack/gguf-parser-go v0.23.1
github.com/gpustack/gguf-parser-go v0.24.0
github.com/hpcloud/tail v1.0.0
github.com/ipfs/go-log v1.0.5
github.com/jaypipes/ghw v0.23.0
@@ -33,7 +31,7 @@ require (
github.com/mholt/archiver/v3 v3.5.1
github.com/microcosm-cc/bluemonday v1.0.27
github.com/modelcontextprotocol/go-sdk v1.3.0
github.com/mudler/cogito v0.9.1-0.20260217143801-bb7f986ed2c7
github.com/mudler/cogito v0.9.1
github.com/mudler/edgevpn v0.31.1
github.com/mudler/go-processmanager v0.1.0
github.com/mudler/memory v0.0.0-20251216220809-d1256471a6c2
@@ -52,7 +50,7 @@ require (
github.com/shirou/gopsutil/v3 v3.24.5
github.com/streamer45/silero-vad-go v0.2.1
github.com/stretchr/testify v1.11.1
github.com/swaggo/echo-swagger v1.4.1
github.com/swaggo/echo-swagger v1.5.0
github.com/swaggo/swag v1.16.6
github.com/testcontainers/testcontainers-go v0.40.0
github.com/tmc/langchaingo v0.1.14
@@ -66,9 +64,11 @@ require (
)
require (
github.com/ghodss/yaml v1.0.0 // indirect
github.com/labstack/echo/v5 v5.0.0 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/sv-tools/openapi v0.2.1 // indirect
github.com/swaggo/files/v2 v2.0.2 // indirect
github.com/swaggo/swag/v2 v2.0.0-rc4 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
@@ -76,6 +76,7 @@ require (
github.com/valyala/fasttemplate v1.2.2 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
require (
@@ -101,7 +102,7 @@ require (
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20240506104042-037f3cc74f2a // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-text/render v0.2.0 // indirect
github.com/go-text/typesetting v0.2.1 // indirect
github.com/go-text/typesetting v0.3.3 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/google/jsonschema-go v0.4.2 // indirect
github.com/hack-pad/go-indexeddb v0.3.2 // indirect

39
go.sum
View File

@@ -8,8 +8,8 @@ dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3/go.mod h1:Yl
dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0/go.mod h1:JLBrvjyP0v+ecvNYvCpyZgu5/xkfAUhi6wJj28eUfSU=
dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412/go.mod h1:a1inKt/atXimZ4Mv927x+r7UpyzRUf4emIoiiSC2TN4=
dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c/go.mod h1:0PRwlb0D6DFvNNtx+9ybjezNCa8XF0xaYcETyp6rHWU=
fyne.io/fyne/v2 v2.7.2 h1:XiNpWkn0PzX43ZCjbb0QYGg1RCxVbugwfVgikWZBCMw=
fyne.io/fyne/v2 v2.7.2/go.mod h1:PXbqY3mQmJV3J1NRUR2VbVgUUx3vgvhuFJxyjRK/4Ug=
fyne.io/fyne/v2 v2.7.3 h1:xBT/iYbdnNHONWO38fZMBrVBiJG8rV/Jypmy4tVfRWE=
fyne.io/fyne/v2 v2.7.3/go.mod h1:gu+dlIcZWSzKZmnrY8Fbnj2Hirabv2ek+AKsfQ2bBlw=
fyne.io/systray v1.12.0 h1:CA1Kk0e2zwFlxtc02L3QFSiIbxJ/P0n582YrZHT7aTM=
fyne.io/systray v1.12.0/go.mod h1:RVwqP9nYMo7h5zViCBHri2FgjXF7H2cub7MAq4NSoLs=
git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999/go.mod h1:fPE2ZNJGynbRyZ4dJvy6G277gSllfV2HJqblrnkyeyg=
@@ -44,8 +44,8 @@ github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
github.com/anthropics/anthropic-sdk-go v1.22.0 h1:sgo4Ob5pC5InKCi/5Ukn5t9EjPJ7KTMaKm5beOYt6rM=
github.com/anthropics/anthropic-sdk-go v1.22.0/go.mod h1:WTz31rIUHUHqai2UslPpw5CwXrQP3geYBioRV4WOLvE=
github.com/anthropics/anthropic-sdk-go v1.26.0 h1:oUTzFaUpAevfuELAP1sjL6CQJ9HHAfT7CoSYSac11PY=
github.com/anthropics/anthropic-sdk-go v1.26.0/go.mod h1:qUKmaW+uuPB64iy1l+4kOSvaLqPXnHTTBKH6RVZ7q5Q=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
github.com/aymanbagabas/go-udiff v0.2.0 h1:TK0fH4MteXUDspT88n8CKzvK0X9O2xu9yQjWpi6yML8=
@@ -128,6 +128,8 @@ github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5Qvfr
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/docker/cli v29.0.3+incompatible h1:8J+PZIcF2xLd6h5sHPsp5pvvJA+Sr2wGQxHkRl53a1E=
github.com/docker/cli v29.0.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
@@ -176,7 +178,6 @@ github.com/fyne-io/image v0.1.1 h1:WH0z4H7qfvNUw5l4p3bC1q70sa5+YWVt6HCj7y4VNyA=
github.com/fyne-io/image v0.1.1/go.mod h1:xrfYBh6yspc+KjkgdZU/ifUC9sPA5Iv7WYUBzQKK7JM=
github.com/fyne-io/oksvg v0.2.0 h1:mxcGU2dx6nwjJsSA9PCYZDuoAcsZ/OuJlvg/Q9Njfo8=
github.com/fyne-io/oksvg v0.2.0/go.mod h1:dJ9oEkPiWhnTFNCmRgEze+YNprJF7YRbpjgpWS4kzoI=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gkampitakis/ciinfo v0.3.2 h1:JcuOPk8ZU7nZQjdUhctuhQofk7BGHuIy0c9Ez8BNhXs=
github.com/gkampitakis/ciinfo v0.3.2/go.mod h1:1NIwaOcFChN4fa/B0hEBdAb6npDlFL8Bwx4dfRLRqAo=
@@ -218,10 +219,10 @@ github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1v
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-text/render v0.2.0 h1:LBYoTmp5jYiJ4NPqDc2pz17MLmA3wHw1dZSVGcOdeAc=
github.com/go-text/render v0.2.0/go.mod h1:CkiqfukRGKJA5vZZISkjSYrcdtgKQWRa2HIzvwNN5SU=
github.com/go-text/typesetting v0.2.1 h1:x0jMOGyO3d1qFAPI0j4GSsh7M0Q3Ypjzr4+CEVg82V8=
github.com/go-text/typesetting v0.2.1/go.mod h1:mTOxEwasOFpAMBjEQDhdWRckoLLeI/+qrQeBCTGEt6M=
github.com/go-text/typesetting-utils v0.0.0-20241103174707-87a29e9e6066 h1:qCuYC+94v2xrb1PoS4NIDe7DGYtLnU2wWiQe9a1B1c0=
github.com/go-text/typesetting-utils v0.0.0-20241103174707-87a29e9e6066/go.mod h1:DDxDdQEnB70R8owOx3LVpEFvpMK9eeH1o2r0yZhFI9o=
github.com/go-text/typesetting v0.3.3 h1:ihGNJU9KzdK2QRDy1Bm7FT5RFQoYb+3n3EIhI/4eaQc=
github.com/go-text/typesetting v0.3.3/go.mod h1:vIRUT25mLQaSh4C8H/lIsKppQz/Gdb8Pu/tNwpi52ts=
github.com/go-text/typesetting-utils v0.0.0-20250618110550-c820a94c77b8 h1:4KCscI9qYWMGTuz6BpJtbUSRzcBrUSSE0ENMJbNSrFs=
github.com/go-text/typesetting-utils v0.0.0-20250618110550-c820a94c77b8/go.mod h1:3/62I4La/HBRX9TcTpBj4eipLiwzf+vhI+7whTc9V7o=
github.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=
github.com/goccy/go-yaml v1.18.0 h1:8W7wMFS12Pcas7KU+VVkaiCng+kG8QiFeFwzFb+rwuw=
github.com/goccy/go-yaml v1.18.0/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
@@ -294,8 +295,8 @@ github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gpustack/gguf-parser-go v0.23.1 h1:0U7DOrsi7ryx2L/dlMy+BSQ5bJV4AuMEIgGBs4RK46A=
github.com/gpustack/gguf-parser-go v0.23.1/go.mod h1:y4TwTtDqFWTK+xvprOjRUh+dowgU2TKCX37vRKvGiZ0=
github.com/gpustack/gguf-parser-go v0.24.0 h1:tdJceXYp9e5RhE9RwVYIuUpir72Jz2D68NEtDXkKCKc=
github.com/gpustack/gguf-parser-go v0.24.0/go.mod h1:y4TwTtDqFWTK+xvprOjRUh+dowgU2TKCX37vRKvGiZ0=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/grpc-ecosystem/grpc-gateway v1.5.0/go.mod h1:RSKVYQBd5MCa4OVpNdGskqpgL2+G+NZTnrVHpWWfpdw=
github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4M0+kPpLofRdBo=
@@ -392,6 +393,8 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/labstack/echo/v4 v4.15.0 h1:hoRTKWcnR5STXZFe9BmYun9AMTNeSbjHi2vtDuADJ24=
github.com/labstack/echo/v4 v4.15.0/go.mod h1:xmw1clThob0BSVRX1CRQkGQ/vjwcpOMjQZSZa9fKA/c=
github.com/labstack/echo/v5 v5.0.0 h1:JHKGrI0cbNsNMyKvranuY0C94O4hSM7yc/HtwcV3Na4=
github.com/labstack/echo/v5 v5.0.0/go.mod h1:SyvlSdObGjRXeQfCCXW/sybkZdOOQZBmpKF0bvALaeo=
github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0=
github.com/labstack/gommon v0.4.2/go.mod h1:QlUFxVM+SNXhDL/Z7YhocGIBYOiwB0mXm1+1bAPHPyU=
github.com/libp2p/go-buffer-pool v0.1.0 h1:oK4mSFcQz7cTQIfqbe4MIj9gLW+mnanjyFtc6cdF0Y8=
@@ -509,10 +512,8 @@ github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7P
github.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/mudler/cogito v0.8.2-0.20260214201734-da0d4ceb2b44 h1:joGszpItINnZdoL/0p2077Wz2xnxMGRSRgYN5mS7I4c=
github.com/mudler/cogito v0.8.2-0.20260214201734-da0d4ceb2b44/go.mod h1:6sfja3lcu2nWRzEc0wwqGNu/eCG3EWgij+8s7xyUeQ4=
github.com/mudler/cogito v0.9.1-0.20260217143801-bb7f986ed2c7 h1:z3AcM7LbaQb+C955JdSXksHB9B0uWGQpdgl05gJM+9Y=
github.com/mudler/cogito v0.9.1-0.20260217143801-bb7f986ed2c7/go.mod h1:6sfja3lcu2nWRzEc0wwqGNu/eCG3EWgij+8s7xyUeQ4=
github.com/mudler/cogito v0.9.1 h1:6y7VPHSS+Q+v4slV42XcjykN5wip4N7C/rXTwWPBVFM=
github.com/mudler/cogito v0.9.1/go.mod h1:6sfja3lcu2nWRzEc0wwqGNu/eCG3EWgij+8s7xyUeQ4=
github.com/mudler/edgevpn v0.31.1 h1:7qegiDWd0kAg6ljhNHxqvp8hbo/6BbzSdbb7/2WZfiY=
github.com/mudler/edgevpn v0.31.1/go.mod h1:ftV5B0nKFzm4R8vR80UYnCb2nf7lxCRgAALxUEEgCf8=
github.com/mudler/go-piper v0.0.0-20241023091659-2494246fd9fc h1:RxwneJl1VgvikiX28EkpdAyL4yQVnJMrbquKospjHyA=
@@ -761,12 +762,16 @@ github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/swaggo/echo-swagger v1.4.1 h1:Yf0uPaJWp1uRtDloZALyLnvdBeoEL5Kc7DtnjzO/TUk=
github.com/swaggo/echo-swagger v1.4.1/go.mod h1:C8bSi+9yH2FLZsnhqMZLIZddpUxZdBYuNHbtaS1Hljc=
github.com/sv-tools/openapi v0.2.1 h1:ES1tMQMJFGibWndMagvdoo34T1Vllxr1Nlm5wz6b1aA=
github.com/sv-tools/openapi v0.2.1/go.mod h1:k5VuZamTw1HuiS9p2Wl5YIDWzYnHG6/FgPOSFXLAhGg=
github.com/swaggo/echo-swagger v1.5.0 h1:nkHxOaBy0SkbJMtMeXZC64KHSa0mJdZFQhVqwEcMres=
github.com/swaggo/echo-swagger v1.5.0/go.mod h1:TzO363X1ZG/MSbjrG2IX6m65Yd3/zpqh5KM6lPctAhk=
github.com/swaggo/files/v2 v2.0.2 h1:Bq4tgS/yxLB/3nwOMcul5oLEUKa877Ykgz3CJMVbQKU=
github.com/swaggo/files/v2 v2.0.2/go.mod h1:TVqetIzZsO9OhHX1Am9sRf9LdrFZqoK49N37KON/jr0=
github.com/swaggo/swag v1.16.6 h1:qBNcx53ZaX+M5dxVyTrgQ0PJ/ACK+NzhwcbieTt+9yI=
github.com/swaggo/swag v1.16.6/go.mod h1:ngP2etMK5a0P3QBizic5MEwpRmluJZPHjXcMoj4Xesg=
github.com/swaggo/swag/v2 v2.0.0-rc4 h1:SZ8cK68gcV6cslwrJMIOqPkJELRwq4gmjvk77MrvHvY=
github.com/swaggo/swag/v2 v2.0.0-rc4/go.mod h1:Ow7Y8gF16BTCDn8YxZbyKn8FkMLRUHekv1kROJZpbvE=
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA=
github.com/testcontainers/testcontainers-go v0.40.0 h1:pSdJYLOVgLE8YdUY2FHQ1Fxu+aMnb6JfVz1mxk7OeMU=
github.com/testcontainers/testcontainers-go v0.40.0/go.mod h1:FSXV5KQtX2HAMlm7U3APNyLkkap35zNLxukw9oBi/MY=