mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
b00422e45f
* feat(backends): add LongCat video and avatar generation Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] [web] * refactor(config): declare model I/O modalities Make model configs declare input and output modalities so capability discovery no longer branches on backend or checkpoint names. Complete the LongCat gallery and user documentation, make the SDPA patch apply to the pinned upstream revision, and stabilize the Agent Jobs race exposed by the required hook. Assisted-by: Codex:GPT-5 [web] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2.9 KiB
2.9 KiB
+++ disableToc = false title = "Features" weight = 8 icon = "lightbulb" type = "chapter" url = "/features/" +++
LocalAI provides a comprehensive set of features for running AI models locally. This section covers all the capabilities and functionalities available in LocalAI.
Core Features
- Text Generation - Generate text with GPT-compatible models using various backends
- Image Generation - Create images with Stable Diffusion and other diffusion models
- Audio Processing - Transcribe audio to text and generate speech from text
- Text to Audio - Generate speech from text with TTS models
- Sound Generation - Generate music and sound effects from text descriptions
- Voice Activity Detection - Detect speech segments in audio data
- Video Generation - Generate videos from text prompts with optional image or audio conditioning
- LongCat Video and Avatar - Run text/image-to-video and audio-driven avatars on NVIDIA CUDA and DGX Spark
- Embeddings - Generate vector embeddings for semantic search and RAG applications
- GPT Vision - Analyze and understand images with vision-language models
Advanced Features
- OpenAI Functions - Use function calling and tools API with local models
- Realtime API - Low-latency multi-modal conversations (voice+text) over WebSocket
- Constrained Grammars - Control model output format with BNF grammars
- GPU Acceleration - Optimize performance with GPU support
- Distribution - Scale inference across multiple nodes (P2P federation or production distributed mode)
- P2P API - Monitor and manage P2P worker and federated nodes
- Model Context Protocol (MCP) - Enable agentic capabilities with MCP integration
- Agents - Autonomous AI agents with tools, knowledge base, and skills
Specialized Features
- Object Detection - Detect and locate objects in images
- Reranker - Improve retrieval accuracy with cross-encoder models
- Stores - Vector similarity search for embeddings
- Model Gallery - Browse and install pre-configured models
- Backends - Learn about available backends and how to manage them
- Backend Monitor - Monitor backend status and resource usage
- Runtime Settings - Configure application settings via web UI without restarting
Getting Started
To start using these features, make sure you have LocalAI installed and have downloaded some models. Then explore the feature pages above to learn how to use each capability.