* fix: Automatically disable mmap for Intel SYCL backends
Fixes issue #9012 where Qwen3.5 models fail to load on Intel Arc GPU
with RPC EOF error.
The Intel SYCL backend has a known issue where mmap enabled causes
the backend to hang. This change automatically disables mmap when
detecting Intel or SYCL backends.
References:
- https://github.com/mudler/LocalAI/issues/9012
- Documentation mentions: SYCL hangs when mmap: true is set
* feat: Add logging for mmap auto-disable on Intel SYCL backends
As requested in PR review, add xlog.Info call to log when mmap
is automatically disabled for Intel SYCL backends. This helps
with debugging and confirms the auto-disable logic is working.
---------
Co-authored-by: localai-bot <localai-bot@users.noreply.github.com>
AIO images are behind, and takes effort to maintain these. Wizard and
installation of models have been semplified massively, so AIO images
lost their purpose.
This allows us to be more laser focused on main images and reliefes
stress from CI.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(acestep-cpp): resolve relative model paths in options
The acestep-cpp backend was failing to load models because the model
paths in options (text_encoder_model, dit_model, vae_model) were being
passed to the C++ code without resolving their relative paths.
When a user configures acestep-cpp-turbo-4b, the model paths are specified
as relative paths like 'acestep-cpp/acestep-v15-turbo-Q8_0.gguf'. The
backend was passing these paths directly to the C++ code without joining
them with the model directory.
This fix:
1. Gets the base directory from the ModelFile path
2. Resolves all relative paths in options to be absolute paths
3. Adds debug logging to show resolved paths for troubleshooting
Fixes#8991
Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
* Apply suggestion from @mudler
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
* test: fix acestep tests to not join modeldir in options
According to code review feedback, the Options array in TestLoadModel
and TestSoundGeneration should contain just the model filenames without
filepath.Join with modelDir. The model paths are handled internally by
the backend.
* fix: change bpm parameter type to float32 to match C++ API signature
* test: fix TestLoadModel and TestSoundGeneration to use baseDir for model paths
- Modified TestLoadModel to compute baseDir from main model path and use it for relative model paths
- Modified TestSoundGeneration similarly to use baseDir for model paths
- Changed bpm parameter type from int32 to float32 to match C++ API
* Apply suggestions from code review
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
---------
Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: localai-bot <localai-bot@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Also test for regressions in HTTP GET API key exempted endpoints because
this list can get out of sync with the UI routes.
Also fix support for proxying on a different prefix both server and
client side.
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(realtime): WebRTC support
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(tracing): Show full LLM opts and deltas
Signed-off-by: Richard Palethorpe <io@richiejp.com>
---------
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* Remove HuggingFace backend support, restore other backends
- Removed backend/go/huggingface directory and all related files
- Removed pkg/langchain/huggingface.go
- Removed LCHuggingFaceBackend from pkg/model/initializers.go
- Removed huggingface backend entries from backend/index.yaml
- Updated backend/README.md to remove HuggingFace backend reference
- Restored kitten-tts, local-store, silero-vad, piper backends that were incorrectly removed
This change removes only HuggingFace backend support from LocalAI
as per the P0 priority request in issue #8963, while preserving
other backends (kitten-tts, local-store, silero-vad, piper).
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
* Remove huggingface backend from test.yml build command
The tests-linux CI job was failing because it was trying to build the
huggingface backend which no longer exists after the backend removal.
This removes huggingface from the build command in test.yml.
* Apply suggestion from @mudler
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
---------
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Otherwise if using collections with postgresql we create a deadlock, as
we need embeddings to be up
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Removed deprecated Qwen3.5-35B-A3B model configuration and updated model path for Qwen3.
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
* docs: Add documentation about GPU auto-fit mode limitations (closes#8562)
- Document the default gpu_layers behavior (9999999) that disables auto-fit
- Explain the trade-off between auto-fit and VRAM threshold unloading
- Add recommendations for users who want to enable gpu_layers: -1
- Note known issues with tensor_buft_override buffer errors
- Link to issue #8562 for future improvements
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
* Apply suggestion from @mudler
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
---------
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Container images that install CUDA runtime libraries (e.g., cuda-cudart-12-5
via apt) create /usr/local/cuda-12 directories as a side effect. The previous
code checked for these directories before checking whether a GPU was present,
causing CPU-only hosts to select a CUDA backend that crashes because
libcuda.so.1 is absent.
Reorder checks so CUDA directory existence only refines the capability when
an NVIDIA GPU is actually detected, consistent with the arm64 L4T code path.
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
* fix: include model name in mmproj file path to prevent model isolation issues
This fix addresses issue #8937 where different models with mmproj files
having the same filename (e.g., mmproj-F32.gguf) would overwrite each other.
By including the model name in the path (llama-cpp/mmproj/<model-name>/<filename>),
each model's mmproj files are now stored in separate directories, preventing
the collision that caused conversations to fail when switching between models.
Fixes#8937
Signed-off-by: LocalAI Bot <localai-bot@example.com>
* test: update test expectations for model name in mmproj path
The test file had hardcoded expectations for the old mmproj path format.
Updated the test expectations to include the model name subdirectory
to match the new path structure introduced in the fix.
Fixes CI failures on tests-apple and tests-linux
* fix: add model name to model path for consistency with mmproj path
This change makes the model path consistent with the mmproj path by
including the model name subdirectory in both paths:
- mmproj: llama-cpp/mmproj/<model-name>/<filename>
- model: llama-cpp/models/<model-name>/<filename>
This addresses the reviewer's feedback that the model config generation
needs to correctly reference the mmproj file path.
Fixes the issue where the model path didn't include the model name
subdirectory while the mmproj path did.
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
---------
Signed-off-by: LocalAI Bot <localai-bot@example.com>
Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Co-authored-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
* fix: add missing bufio.Flush in processImageFile
The processImageFile function writes decoded image data (from base64
or URL download) through a bufio.NewWriter but never calls Flush()
before closing the underlying file. Since bufio's default buffer is
4096 bytes, small images produce 0-byte files and large images are
truncated — causing PIL to fail with "cannot identify image file".
This breaks all image input paths: file, files, and ref_images
parameters in /v1/images/generations, making img2img, inpainting,
and reference image features non-functional.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
* fix: merge options into kwargs in diffusers GenerateImage
The GenerateImage method builds a local `options` dict containing the
source image (PIL), negative_prompt, and num_inference_steps, but
never merges it into `kwargs` before calling self.pipe(**kwargs).
This causes img2img to fail with "Input is in incorrect format"
because the pipeline never receives the image parameter.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
* test: add unit test for processImageFile base64 decoding
Verifies that a base64-encoded PNG survives the write path
(encode → decode → bufio.Write → Flush → file on disk) with
byte-for-byte fidelity. The test image is small enough to fit
entirely in bufio's 4096-byte buffer, which is the exact scenario
where the missing Flush() produced a 0-byte file.
Also tests that invalid base64 input is handled gracefully.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
* test: verify GenerateImage merges options into pipeline kwargs
Mocks the diffusers pipeline and calls GenerateImage with a source
image and negative prompt. Asserts that the pipeline receives the
image, negative_prompt, and num_inference_steps via kwargs — the
exact parameters that were silently dropped before the fix.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
* fix: move kwargs.update(options) earlier in GenerateImage
Move the options merge right after self.options merge (L742) so that
image, negative_prompt, and num_inference_steps are available to all
downstream code paths including img2vid and txt2vid.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
* test: convert processImageFile tests to ginkgo
Replace standard testing with ginkgo/gomega to be consistent with
the rest of the test suites in the project.
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
---------
Signed-off-by: Attila Györffy <attila+git@attilagyorffy.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
