mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-13 03:09:03 -04:00
085fc53bbc
* fix(router): score classifier production-readiness Conversation trimming runs through the classifier model's chat template and trims by exact token count, sized to the model's n_batch which is now scaled to context so long probes can't crash the backend. Missing chat_message templates are a hard error at router build time. Router- facing factories (Embedder/Scorer/Reranker/TokenCounter) re-resolve ModelConfig per call so a model installed post-startup doesn't bind a stub Backend="" config and silently fall into the loader's auto- iterate path. New 'vector_store' backend trace recorded inside localVectorStore on every Search/Insert — including the backend-load-failure path that previously vanished into an xlog.Warn — with outcome tagging (hit/miss/empty_store/backend_load_error/find_error/insert_error/ok). Companion cleanup drops misleading similarity:0 and input_tokens_count:0 from non-hit and text-mode traces. Gallery local-store-development aliases to 'local-store' so the master image satisfies pkg/model.LocalStoreBackend lookups from the embedding cache. Misc: llama-cpp TokenizeString reads the correct 'prompt' JSON key (the original bug); ModelTokenize nil-guard; non-fatal mitm proxy startup; PII 'route_local' renamed to 'allow' with docs/UI in sync; model-editor footer no longer eats the edit area on small screens; several config-editor template/dropdown/section fixes. Tests: e2e router specs (casual/code-hint + long-conversation trim), vector_store trace specs, lazy-factory specs, gallery dev-alias resolution, Playwright trace badge + scroll regression. Assisted-by: Claude:claude-opus-4-7 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(backend): auto-size batch to context for embedding and rerank models Embedding and rerank models pool over the whole input in a single physical batch (n_ubatch). With batch left at the 512 default, the backend rejects longer inputs with "input is too large to process", silently capping a large-context embedder (e.g. 8k/32k) at 512 tokens. Size n_batch to the context for these single-pass usecases, mirroring the existing FLAG_SCORE behaviour; an explicit batch: still wins. Extracts EffectiveContextSize/EffectiveBatchSize from grpcModelOpts so the effective decode window has one home for other callers to reuse. Adds an e2e-aio regression test that embeds a >512-token input. The AIO embedding model is switched to nomic-embed-text-v1.5 (2048 context) because the previous granite model was capped at 512 tokens and could not exercise the larger batch. Assisted-by: claude-code:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(gallery): raise arch-router scoring output cap via parallel:64 Scoring decodes the whole prompt+candidate in a single llama_decode and reads one logit row per candidate token. The vendored llama.cpp server caps causal output rows at n_parallel, so the default of 1 aborts with GGML_ASSERT(n_outputs_max <= cparams.n_outputs_max) on multi-token route labels. Set options: [parallel:64] on both arch-router quant entries to lift the cap; kv_unified (the grpc-server default) keeps the full context per sequence, so this does not split the KV cache. Assisted-by: claude-code:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>
40 lines
1.5 KiB
JavaScript
40 lines
1.5 KiB
JavaScript
import { defineConfig } from '@playwright/test'
|
|
|
|
export default defineConfig({
|
|
testDir: './e2e',
|
|
timeout: 30_000,
|
|
retries: process.env.CI ? 2 : 0,
|
|
// TEMPORARY: cap parallelism. Playwright's default (cores/2) oversubscribes
|
|
// high-core dev machines and intermittently starves the page-teardown
|
|
// coverage harvest past the 30s test timeout (flaky "Tearing down page"
|
|
// failures, different specs each run). Capped at 8 pending a proper
|
|
// root-cause fix; override with PW_WORKERS.
|
|
workers: process.env.PW_WORKERS ? Number(process.env.PW_WORKERS) : 8,
|
|
reporter: process.env.CI ? 'html' : 'list',
|
|
use: {
|
|
baseURL: 'http://127.0.0.1:8089',
|
|
trace: 'on-first-retry',
|
|
},
|
|
projects: [
|
|
{
|
|
name: 'chromium',
|
|
use: {
|
|
browserName: 'chromium',
|
|
// Use a nix-provided Chromium when PLAYWRIGHT_CHROMIUM_PATH is set
|
|
// (the flake dev shell exports it). Avoids Playwright's downloaded
|
|
// browser, which can't resolve system libs (libglib-2.0, …) on NixOS.
|
|
// Unset in CI, where `playwright install --with-deps` is used instead.
|
|
...(process.env.PLAYWRIGHT_CHROMIUM_PATH
|
|
? { launchOptions: { executablePath: process.env.PLAYWRIGHT_CHROMIUM_PATH } }
|
|
: {}),
|
|
},
|
|
},
|
|
],
|
|
webServer: process.env.PLAYWRIGHT_EXTERNAL_SERVER ? undefined : {
|
|
command: '../../../tests/e2e-ui/ui-test-server --mock-backend=../../../tests/e2e/mock-backend/mock-backend --port=8089 > /tmp/ui-test-server.log 2>&1',
|
|
port: 8089,
|
|
timeout: 120_000,
|
|
reuseExistingServer: !process.env.CI,
|
|
},
|
|
})
|