mirror of
https://github.com/mudler/LocalAI.git
synced 2026-02-05 12:12:39 -05:00
dd8e74a486
* feat(realtime): Add audio conversations Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(realtime): Vendor the updated API and modify for server side Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): Update to the GA realtime API Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore: Document realtime API and add docs to AGENTS.md Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat: Filter reasoning from spoken output Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): Send delta and done events for tool calls and audio transcripts Ensure that content is sent in both deltas and done events for function call arguments and audio transcripts. This fixes compatibility with clients that rely on delta events for parsing. 💘 Generated with Crush Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): Improve tool call handling and error reporting - Refactor Model interface to accept []types.ToolUnion and *types.ToolChoiceUnion instead of JSON strings, eliminating unnecessary marshal/unmarshal cycles - Fix Parameters field handling: support both map[string]any and JSON string formats - Add PredictConfig() method to Model interface for accessing model configuration - Add comprehensive debug logging for tool call parsing and function config - Add missing return statement after prediction error (critical bug fix) - Add warning logs for NoAction function argument parsing failures - Improve error visibility throughout generateResponse function 💘 Generated with Crush Assisted-by: Claude Sonnet 4.5 via Crush <crush@charm.land> Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>
2.1 KiB
2.1 KiB
+++ disableToc = false title = "Features" weight = 8 icon = "lightbulb" type = "chapter" url = "/features/" +++
LocalAI provides a comprehensive set of features for running AI models locally. This section covers all the capabilities and functionalities available in LocalAI.
Core Features
- Text Generation - Generate text with GPT-compatible models using various backends
- Image Generation - Create images with Stable Diffusion and other diffusion models
- Audio Processing - Transcribe audio to text and generate speech from text
- Embeddings - Generate vector embeddings for semantic search and RAG applications
- GPT Vision - Analyze and understand images with vision-language models
Advanced Features
- OpenAI Functions - Use function calling and tools API with local models
- Realtime API - Low-latency multi-modal conversations (voice+text) over WebSocket
- Constrained Grammars - Control model output format with BNF grammars
- GPU Acceleration - Optimize performance with GPU support
- Distributed Inference - Scale inference across multiple nodes
- Model Context Protocol (MCP) - Enable agentic capabilities with MCP integration
Specialized Features
- Object Detection - Detect and locate objects in images
- Reranker - Improve retrieval accuracy with cross-encoder models
- Stores - Vector similarity search for embeddings
- Model Gallery - Browse and install pre-configured models
- Backends - Learn about available backends and how to manage them
- Runtime Settings - Configure application settings via web UI without restarting
Getting Started
To start using these features, make sure you have LocalAI installed and have downloaded some models. Then explore the feature pages above to learn how to use each capability.