Files
LocalAI/flake.nix
Richard Palethorpe 718223f33b feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113)
* chore(localvqe): update backend to v1.3, add v1.2/v1.3 gallery models

Bump the LocalVQE backend pin 72bfb4c6 -> b0f0378a, which adds the v1.2
(1.3 M) and v1.3 (4.8 M) GGUF SHA-256s to the upstream released-models
allowlist (and the arch_version=3 loader) so both load without
LOCALVQE_ALLOW_UNHASHED.

Add gallery entries for localvqe-v1.2-1.3m and localvqe-v1.3-4.8m
(SHA-256 verified against the downloaded weights) and update the
audio-transform docs to make v1.3 the current default while noting the
compact v1.1/v1.2 alternatives.

Assisted-by: Claude:claude-opus-4-8 Claude-Code
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(flake): add ffmpeg-headless to the dev shell

pkg/utils/ffmpeg_test.go shells out to the `ffmpeg` CLI, and the
pre-commit gate runs those tests via `make test-coverage`. Without
ffmpeg in the dev shell the gate fails with "executable file not found
in $PATH". The headless build provides the CLI without GUI/X deps.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(localvqe): parse WAV by walking RIFF sub-chunks

Walk the RIFF chunk list instead of assuming the canonical 44-byte
header layout. Real inputs (browser-recorded clips, ffmpeg output with
an 18/40-byte extensible `fmt ` chunk or trailing LIST/INFO metadata)
would otherwise splice header/metadata bytes into the PCM stream as an
audible impulse. Honour the `data` chunk size and validate that both
`fmt ` and `data` chunks are present.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(security-headers): allow blob: in connect-src for waveform fetch

The waveform renderer XHRs/fetches a freshly-created blob: object URL
(e.g. an uploaded or enhanced clip before it has a server URL). XHR/fetch
of blob: is governed by connect-src, not media-src, so it was blocked by
the CSP. Add blob: to connect-src.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(react-ui): add input/output spectrogram view to AudioTransform

The transform page only showed time-domain amplitude waveforms, so you
could see how loud a clip was but not which frequencies the model
touched. Add a time x frequency spectrogram heatmap and render the input
and output spectrums side by side, so it's visible which bands the
enhancement attenuates (bright input bands that go dark in the output).

Computed client-side via a Hann-windowed STFT over both clips (a small
dependency-free radix-2 FFT), defaulting to the LocalVQE 512/256 frame
geometry. This shows the net input->output spectral change; the model's
internal gain mask is not exposed by the backend.

- src/utils/fft.js            radix-2 FFT
- src/hooks/useSpectrogram.js decode + STFT -> normalised dB magnitude grid
- src/components/audio/Spectrogram.jsx  canvas heatmap (magma colormap)
- AudioTransform.jsx          dual-spectrogram panel + CSS
- e2e spec + UI coverage baseline bump (38.29 -> 39.0; measured ~39.4-40.2)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test(react-ui): make UI coverage deterministic, tighten the gate

UI e2e line coverage swung ~1pp run-to-run (39.1% <-> 40.2%), which forced
a loose 0.8pp tolerance on the monotonic gate — a band wide enough to let
a real ~300-line regression through silently. The swing was a bug, not
inherent jitter: the 'Create Agent navigates' spec ended on the URL
assertion, so AgentCreate.jsx's ~400 lines were collected only when its
render happened to beat the coverage teardown.

Wait for the page to actually render (assert its heading) so those lines
are covered every run. With the race gone, repeated runs land within
~0.013pp of each other, so:

- tighten UI_COVERAGE_TOLERANCE 0.8 -> 0.1 (noise floor, not a drift band)
- set the baseline to the real, reliably-achieved value (39.0 -> 39.86)

Localised by running the V8-coverage suite repeatedly and diffing per-file
line coverage; AgentCreate.jsx was the sole ~1pp flipper.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-05-31 23:56:46 +02:00

111 lines
3.6 KiB
Nix

# Made by Azteczek
{
description = "LocalAI flake";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
inference-defaults = {
url = "https://raw.githubusercontent.com/unslothai/unsloth/main/studio/backend/assets/configs/inference_defaults.json";
flake = false;
};
};
outputs = { self, nixpkgs, inference-defaults }:
let
system = "x86_64-linux";
pkgs = nixpkgs.legacyPackages.${system};
in {
packages.${system}.default = pkgs.buildGoModule {
pname = "localai";
version = "custom";
src = ./.;
proxyVendor = true;
vendorHash = "sha256-6f3adjGsoFXlUtXjBDHP4Mv9jKCOK3aeUXprm0EAVO8=";
nativeBuildInputs = with pkgs; [
pkg-config cmake gcc protobuf go-protobuf protoc-gen-go protoc-gen-go-grpc
];
env = {
CGO_ENABLED = "0";
};
preBuild = ''
PROTO_SOURCE_DIR=$(find . -name "*.proto" -printf "%h" -quit)
mkdir -p pkg/grpc/proto
${pkgs.protobuf}/bin/protoc \
-I=$PROTO_SOURCE_DIR \
-I. \
--go_out=pkg/grpc/proto --go_opt=paths=source_relative \
--go-grpc_out=pkg/grpc/proto --go-grpc_opt=paths=source_relative \
$PROTO_SOURCE_DIR/*.proto
go mod edit -replace github.com/mudler/LocalAI/pkg/grpc/proto=./pkg/grpc/proto
mkdir -p core/config/gen_inference_defaults
cp ${inference-defaults} core/config/gen_inference_defaults/inference_defaults.json
sed -i '/go:generate/d' core/config/inference_defaults.go || true
'';
subPackages = [ "cmd/local-ai" ];
doCheck = false;
postInstall = ''
[ -f $out/bin/local-ai ] && mv $out/bin/local-ai $out/bin/localai
'';
};
devShells.${system}.default = pkgs.mkShell {
packages = with pkgs; [
# Build toolchain (stdenv already provides gcc)
go
gnumake
pkg-config
cmake
protobuf
go-protobuf
protoc-gen-go
protoc-gen-go-grpc
# React UI build (core/http/react-ui — `make react-ui`)
nodejs
bun # alternative to npm, used by `make react-ui-docker`
chromium # Playwright e2e / UI coverage browser (see PLAYWRIGHT_CHROMIUM_PATH below)
# Linting / static analysis (see `make lint`)
golangci-lint
gofumpt
gotools # goimports
go-tools # staticcheck
# Audio transforms: pkg/utils/ffmpeg_test.go shells out to the
# `ffmpeg` CLI, exercised by `make test-coverage` (the pre-commit
# gate). Headless build = the CLI without GUI/X deps.
ffmpeg-headless
# Common dev conveniences
git
curl
];
shellHook = ''
# Point Playwright at the nix-provided Chromium instead of its own
# downloaded build, which can't resolve system libs (libglib-2.0, )
# on NixOS. playwright.config.js reads PLAYWRIGHT_CHROMIUM_PATH and
# the Makefile skips `playwright install` when it's set.
export PLAYWRIGHT_CHROMIUM_PATH="${pkgs.chromium}/bin/chromium"
export PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
echo "LocalAI dev shell: $(go version), node $(node --version)"
echo "Build: make build (Go binary + React UI)"
echo "React UI: make react-ui (npm install && vite build)"
echo "Lint: make lint (only new issues vs master)"
echo " or make lint-all (full baseline)"
'';
};
};
}