mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-12 22:33:54 -04:00
docs(website): simplify installation paths (#11631)
Keep the homepage focused on runtime capabilities and move engine details to their canonical directory. Make installation choices stable and explicit for users across supported hardware. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-bot <306113404+localai-org-bot@users.noreply.github.com>
This commit is contained in:
1 parent
a0252ad6a1
commit
3684a534bb
5 files changed
+91
-148
No files matched your search
Executable
+44
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
python3 - <<'PY'
|
||||
from pathlib import Path
|
||||
|
||||
home = Path("website/layouts/index.html").read_text()
|
||||
css = Path("website/static/css/site.css").read_text()
|
||||
install = Path("docs/content/getting-started/install.md").read_text()
|
||||
containers = Path("docs/content/getting-started/containers.md").read_text()
|
||||
|
||||
def require(condition, message):
|
||||
if not condition:
|
||||
raise SystemExit(f"FAIL: {message}")
|
||||
|
||||
require("Drop-in replacement for most upstream APIs." in home,
|
||||
"homepage must use the requested drop-in API heading")
|
||||
require("Everything else plugs into LocalAI." not in home,
|
||||
"old runtime heading must be removed")
|
||||
require("When the engine we need" not in home,
|
||||
"hero must describe user outcomes instead of team implementation")
|
||||
require('href="mailto:contact@localai.io"' in home and "business" in home.lower(),
|
||||
"homepage must provide a direct business contact action")
|
||||
require(home.index('id="localai"') < home.index('id="proof-quotes"') < home.index('id="mission"'),
|
||||
"headline testimonials must directly follow the runtime section")
|
||||
require(home.count('id="proof-quotes"') == 1,
|
||||
"headline testimonials must appear exactly once")
|
||||
require('id="engines"' not in home and "Engines we build" not in home,
|
||||
"homepage engine showcase must be removed")
|
||||
require('href="/docs/installation/index.html"' in home,
|
||||
"installation guide action must use the direct installation URL")
|
||||
require('<iframe' in install and "youtube.com/embed/cMVNnlqwfw4" in install,
|
||||
"installation page must embed the walkthrough video")
|
||||
require("## Quick Start" not in install,
|
||||
"installation landing page must not duplicate Quick Start")
|
||||
for text in ("CUDA 12", "CUDA 13", "ROCm", "Intel", "Jetson", "Vulkan", "fallback"):
|
||||
require(text.lower() in containers.lower(), f"GPU chooser must explain {text}")
|
||||
require('class="sn__e"><a href="https://github.com/mudler/parakeet.cpp">parakeet.cpp</a>' in home,
|
||||
"capability engine names must link to their repositories")
|
||||
require(".pane{min-height:" in css.replace(" ", ""),
|
||||
"all installation panes must have a fixed minimum height")
|
||||
|
||||
print("website review 143 source checks passed")
|
||||
PY
|
||||
@@ -48,6 +48,13 @@ podman run -ti --name local-ai -p 8080:8080 localai/localai:latest
|
||||
|
||||
#### GPU Images
|
||||
|
||||
Choose the image that matches your hardware and installed drivers:
|
||||
|
||||
- **NVIDIA CUDA 12** is the compatibility choice for systems with CUDA 12 drivers. Use **CUDA 13** when your NVIDIA driver and toolkit support CUDA 13.
|
||||
- **AMD ROCm** is for supported AMD GPUs, while **Intel** is for Intel GPUs with the required device runtime.
|
||||
- **Jetson** uses the L4T ARM64 image. Choose its CUDA 12 image for Jetson AGX Orin-class devices or CUDA 13 for DGX Spark.
|
||||
- **Vulkan** works across vendors and is the fallback when no matching CUDA, ROCm, or Intel image is available.
|
||||
|
||||
**NVIDIA CUDA 13:**
|
||||
```bash
|
||||
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13
|
||||
|
||||
@@ -10,7 +10,9 @@ LocalAI can be installed in multiple ways depending on your platform and prefere
|
||||
|
||||
## Video Walkthrough
|
||||
|
||||
[](https://www.youtube.com/watch?v=cMVNnlqwfw4)
|
||||
<div class="video-container">
|
||||
<iframe src="https://www.youtube.com/embed/cMVNnlqwfw4" title="LocalAI installation walkthrough" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
|
||||
</div>
|
||||
|
||||
## Installation Methods
|
||||
|
||||
@@ -21,23 +23,3 @@ Choose the installation method that best suits your needs:
|
||||
3. **[Linux]({{% relref "getting-started/linux" %}})** - Install on Linux using binaries
|
||||
4. **[Kubernetes]({{% relref "getting-started/kubernetes" %}})** - Deploy LocalAI on Kubernetes clusters
|
||||
5. **[Build from Source]({{% relref "getting-started/build" %}})** - Build LocalAI from source code
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Recommended: Containers (Docker or Podman)**
|
||||
|
||||
```bash
|
||||
# With Docker
|
||||
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest
|
||||
|
||||
# Or with Podman
|
||||
podman run -p 8080:8080 --name local-ai -ti localai/localai:latest
|
||||
```
|
||||
|
||||
This will start LocalAI. The API will be available at `http://localhost:8080`.
|
||||
|
||||
For other platforms:
|
||||
- **macOS**: Download the [DMG]({{% relref "getting-started/macos" %}})
|
||||
- **Linux**: See the [Linux installation guide]({{% relref "getting-started/linux" %}}) for binary installation.
|
||||
|
||||
For detailed instructions, see the [Containers installation guide]({{% relref "getting-started/containers" %}}).
|
||||
+33
-124
@@ -9,11 +9,12 @@
|
||||
<p class="kicker fd" style="margin-top:0">Open source · MIT · v4.8.0</p>
|
||||
<h1><u><b>Make AI run on</b></u><u><b><s>every machine.</s></b></u></h1>
|
||||
<div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
|
||||
<p class="lede fd mt2">Text, voice, vision, images, video, 3D and agents, from one open runtime. It works on the laptop you already own, and scales to a room full of GPUs when you have one. When the engine we need is too heavy, too closed, or does not exist, we write it.</p>
|
||||
<p class="lede fd mt2">Run text, voice, vision, image, video, 3D and agent workloads through one open runtime. Start on the laptop you already own, keep your data on your hardware, and scale to a room full of GPUs when you need more capacity.</p>
|
||||
<div class="acts fd">
|
||||
<a class="btn" href="#start">Install LocalAI <span>→</span></a>
|
||||
<a class="btn btn--o" href="/docs/">Read the docs</a>
|
||||
<a class="btn btn--o" href="{{ .Site.Params.github }}">★ Star on GitHub</a>
|
||||
<a class="btn btn--o" href="mailto:contact@localai.io">Talk to us about your business</a>
|
||||
</div>
|
||||
<div class="figures fd">
|
||||
<div><b class="tnum" data-count="{{ .Site.Data.stats.stars }}">0</b><span>GitHub stars</span></div>
|
||||
@@ -37,7 +38,7 @@
|
||||
<div class="shell">
|
||||
<div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
|
||||
<p class="kicker rv">The runtime</p>
|
||||
<h2 class="rv mt1" style="max-width:21ch">Everything else plugs into LocalAI.</h2>
|
||||
<h2 class="rv mt1" style="max-width:21ch">Drop-in replacement for most upstream APIs.</h2>
|
||||
<p class="lede rv mt2">One binary with an OpenAI-compatible API in front of it. Point an existing client at it and the calls keep working, except now the model is on your machine. It also speaks the Anthropic, Ollama and ElevenLabs APIs, so most tools need a URL change and nothing else.</p>
|
||||
<p class="lede rv mt2">The engine behind that API is swappable. One model can run on llama.cpp while the next loads on vLLM, SGLang or MLX, and the client never notices: same endpoint, same request, different engine underneath. Switching is one line in the model's config.</p>
|
||||
<p class="lede rv mt2">A small core pulls each engine in as a separate backend, only when a model asks for it. That is why one install covers this much ground without becoming a 9 GB download.</p>
|
||||
@@ -66,6 +67,27 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- HEADLINE TESTIMONIALS -->
|
||||
<section class="navy" id="proof-quotes">
|
||||
<div class="shell">
|
||||
<p class="kicker rv">What other people say</p>
|
||||
<div class="headline">
|
||||
<a class="hq rv" href="https://x.com/ggerganov/status/2065447087311917459">
|
||||
<blockquote>“Some cool ggml-based work by @mudler_it recently, make sure to check it out.”</blockquote>
|
||||
<p class="hq__w">@ggerganov<b>Georgi Gerganov, author of llama.cpp and ggml</b></p>
|
||||
<span class="hq__go">On X, 2026 ↗</span></a>
|
||||
<a class="hq rv" href="https://x.com/badlogicgames/status/2061201400059531729">
|
||||
<blockquote>“What a wonderful project: parakeet.cpp. A ggml based parakeet inference pipeline that is 2x faster than my ONNX parakeet pipeline on Apple Silicon.”</blockquote>
|
||||
<p class="hq__w">@badlogicgames<b>Mario Zechner, author of pi.agent</b></p>
|
||||
<span class="hq__go">On X, 2026 ↗</span></a>
|
||||
<a class="hq rv" href="https://www.linkedin.com/posts/adimargolink_spinoza-saw-all-things-straining-to-become-activity-7468135820076634113-u3Hc">
|
||||
<blockquote>“Build something good enough that the community chooses to carry it beyond your reach. This week, Ettore Di Giacinto brought NVIDIA Parakeet to the CPU.”</blockquote>
|
||||
<p class="hq__w">Adi Margolin<b>On LinkedIn</b></p>
|
||||
<span class="hq__go">Read the post ↗</span></a>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- MISSION -->
|
||||
<section class="navy" id="mission">
|
||||
<div class="shell">
|
||||
@@ -105,28 +127,28 @@
|
||||
<div class="senses">
|
||||
<div class="sn rv"><p class="sn__v">Hear <u>words</u></p>
|
||||
<p>Live transcription with speaker labels and timestamps, fast enough to keep up with a meeting while it is still happening.</p>
|
||||
<div class="sn__e"><span>parakeet.cpp</span><span>moss-transcribe.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/mudler/parakeet.cpp">parakeet.cpp</a><a href="https://github.com/localai-org/moss-transcribe.cpp">moss-transcribe.cpp</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">Hear <u>the room</u></p>
|
||||
<p>527 kinds of sound event: a door, a dog, breaking glass, a smoke alarm. The model notices things nobody thought to type.</p>
|
||||
<div class="sn__e"><span>ced.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/localai-org/ced.cpp">ced.cpp</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">Know <u>who</u></p>
|
||||
<p>Recognise a voice, recognise a face, and tell a live person from a photo held up to the camera.</p>
|
||||
<div class="sn__e"><span>voice-detect.cpp</span><span>face-detect.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/localai-org/voice-detect.cpp">voice-detect.cpp</a><a href="https://github.com/mudler/face-detect.cpp">face-detect.cpp</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">See <u>things</u></p>
|
||||
<p>Ask for "the red mug on the left" in plain language and get back coordinates, not a caption.</p>
|
||||
<div class="sn__e"><span>locate-anything.cpp</span><span>rf-detr.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/mudler/locate-anything.cpp">locate-anything.cpp</a><a href="https://github.com/localai-org/rf-detr.cpp">rf-detr.cpp</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">See <u>space</u></p>
|
||||
<p>Distance in metres from one ordinary photo, and a full 3D reconstruction from a handful of them. No rig, no camera poses, no GPU.</p>
|
||||
<div class="sn__e"><span>depth-anything.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/localai-org/depth-anything.cpp">depth-anything.cpp</a><a href="https://github.com/localai-org/free-splatter.cpp">free-splatter.cpp</a><a href="https://github.com/localai-org/trellis2cpp">trellis2.cpp</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">Speak</p>
|
||||
<p>Long-form speech in a cloned voice, across dozens of languages, up to 48 kHz.</p>
|
||||
<div class="sn__e"><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>vibevoice.cpp</span><span>voxtral-tts.c</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/mudler/moss-tts.cpp">moss-tts.cpp</a><a href="https://github.com/mudler/magpie-tts.cpp">magpie-tts.cpp</a><a href="https://github.com/localai-org/vibevoice.cpp">vibevoice.cpp</a><a href="/engines/">voxtral-tts.c</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">Hear <u>clearly</u></p>
|
||||
<p>Echo cancellation, noise suppression and dereverberation, so a voice loop survives a real room with a real speaker in it.</p>
|
||||
<div class="sn__e"><span>LocalVQE</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/localai-org/LocalVQE">LocalVQE</a></div></div>
|
||||
<div class="sn rv"><p class="sn__v">Forget <u>on purpose</u></p>
|
||||
<p>Names, addresses and card numbers get caught and redacted on the machine, before anything is sent anywhere.</p>
|
||||
<div class="sn__e"><span>privacy-filter.cpp</span></div></div>
|
||||
<div class="sn__e"><a href="https://github.com/localai-org/privacy-filter.cpp">privacy-filter.cpp</a></div></div>
|
||||
</div>
|
||||
<p class="lede rv mt3" style="max-width:74ch;color:var(--ink)">One session can do all of it at once: hear the room, work out who is talking, read what is on the desk, call a tool, and answer out loud. One API, one machine, nothing leaves the building.</p>
|
||||
</div>
|
||||
@@ -160,101 +182,6 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- ENGINES -->
|
||||
<section class="navy" id="engines">
|
||||
<div class="shell">
|
||||
<div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
|
||||
<p class="kicker rv">Engines we build</p>
|
||||
<h2 class="rv mt1" style="max-width:22ch">{{ len .Site.Data.engines.engines }} engines, written from scratch.</h2>
|
||||
{{/* Names the link back to the runtime section explicitly. Readers were
|
||||
arriving here and taking these for unrelated side projects, because
|
||||
nothing on the page said they are the backends the core loads. */}}
|
||||
<p class="lede rv mt2">Those backends the core pulls in on demand are mostly ours. Most projects wrap somebody else's engine. We wrote ours, because the thing we needed came as a 9 GB Python install, or was closed, or did not exist. Each one is a binary and a GGUF file, checked against the reference implementation in CI.</p>
|
||||
|
||||
<div class="spot rv">
|
||||
<div>
|
||||
<h3>parakeet.cpp</h3>
|
||||
<p class="spot__h">Twenty-seven times faster than whisper.cpp, on a CPU.</p>
|
||||
<p>NVIDIA NeMo Parakeet, ported to C++ and ggml. Ten checkpoints, all of them verified at WER 0 against NeMo, which means the transcript is identical to NeMo's while finishing first. Cache-aware streaming with end-of-utterance detection handles live audio, and the multilingual streaming model covers 40 or more locales.</p>
|
||||
<div class="facts">
|
||||
<div><b>27x</b><span>vs whisper.cpp, CPU</span></div>
|
||||
<div><b>1.40x</b><span>vs NeMo, CPU median</span></div>
|
||||
<div><b>WER 0</b><span>Parity with NeMo</span></div>
|
||||
<div><b>37%</b><span>Size at q8_0</span></div>
|
||||
</div>
|
||||
<p class="mt2"><a class="btn btn--o" href="https://github.com/mudler/parakeet.cpp">parakeet.cpp on GitHub ↗</a></p>
|
||||
</div>
|
||||
<div class="spot__m">
|
||||
<figure><video src="/media/parakeet-duel.mp4" muted loop playsinline preload="none" data-lazy aria-label="parakeet.cpp finishing ahead of NeMo on the same audio"></video>
|
||||
<figcaption>Same audio, same words, ours finishes first</figcaption></figure>
|
||||
<figure><img src="/img/parakeet-speedup.jpg" alt="Bar chart of parakeet.cpp CPU speedup over NeMo per dtype" loading="lazy">
|
||||
<figcaption>CPU speedup over NeMo, by dtype</figcaption></figure>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="spot spot--r rv">
|
||||
<div>
|
||||
<h3>depth-anything.cpp</h3>
|
||||
<p class="spot__h">Beats PyTorch on CPU, in half the memory.</p>
|
||||
<p>Depth Anything 3 as a 99 MB file. It gives you metric depth, per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud you can export to glb or COLMAP. Output correlates 1.0 with the reference implementation, component by component, and there is no Python or CUDA toolkit anywhere at inference.</p>
|
||||
<div class="facts">
|
||||
<div><b>1.31x</b><span>vs PyTorch, CPU</span></div>
|
||||
<div><b>363 MB</b><span>Peak RAM, q8_0</span></div>
|
||||
<div><b>6.7x</b><span>Faster to load</span></div>
|
||||
<div><b>99 MB</b><span>Smallest build</span></div>
|
||||
</div>
|
||||
<p class="mt2"><a class="btn btn--o" href="https://github.com/mudler/depth-anything.cpp">depth-anything.cpp on GitHub ↗</a></p>
|
||||
</div>
|
||||
<div class="spot__m">
|
||||
<figure><video src="/media/depth-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="depth-anything.cpp finishing ahead of PyTorch on CPU"></video>
|
||||
<figcaption>One photo in, distance in metres out, ahead of PyTorch on the same CPU</figcaption></figure>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="wallhead rv">
|
||||
<h3>And the rest of them, running.</h3>
|
||||
<p>Every clip comes out of the benchmark suite that runs in CI on that engine. Where you see two panes, ours is racing the reference implementation on the same machine, on the same input.</p>
|
||||
</div>
|
||||
<div class="wall">
|
||||
<a class="wi rv" href="https://github.com/mudler/parakeet.cpp"><video src="/media/parakeet-long.mp4" muted loop playsinline preload="none" data-lazy aria-label="parakeet.cpp racing NeMo on long audio"></video>
|
||||
<div class="wi__c"><em>parakeet.cpp</em> long-form transcription <b>vs NeMo</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/localai-org/ced.cpp"><video src="/media/ced.mp4" muted loop playsinline preload="none" data-lazy aria-label="ced.cpp tagging sound events live"></video>
|
||||
<div class="wi__c"><em>ced.cpp</em> sound events, live <b>527 classes</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/face-detect.cpp"><video src="/media/face.mp4" muted loop playsinline preload="none" data-lazy aria-label="face-detect.cpp detecting, landmarking and recognising faces"></video>
|
||||
<div class="wi__c"><em>face-detect.cpp</em> detect, landmark, recognise <b>no Python</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/face-detect.cpp"><video src="/media/face-id.mp4" muted loop playsinline preload="none" data-lazy aria-label="face-detect.cpp finding the same person in a lineup"></video>
|
||||
<div class="wi__c"><em>face-detect.cpp</em> same person, different photo <b>1 to N</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/localai-org/voice-detect.cpp"><video src="/media/voice.mp4" muted loop playsinline preload="none" data-lazy aria-label="voice-detect.cpp speaker recognition benchmark"></video>
|
||||
<div class="wi__c"><em>voice-detect.cpp</em> who is speaking <b>vs reference</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/depth-anything.cpp"><video src="/media/depth.mp4" muted loop playsinline preload="none" data-lazy aria-label="depth-anything.cpp racing PyTorch on CPU"></video>
|
||||
<div class="wi__c"><em>depth-anything.cpp</em> metric depth <b>vs PyTorch, CPU</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/locate-anything.cpp"><video src="/media/locate.mp4" muted loop playsinline preload="none" data-lazy aria-label="locate-anything.cpp open vocabulary detection race"></video>
|
||||
<div class="wi__c"><em>locate-anything.cpp</em> say it, find it <b>open vocabulary</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/moss-tts.cpp"><video src="/media/moss.mp4" muted loop playsinline preload="none" data-lazy aria-label="moss-tts.cpp synthesis benchmark"></video>
|
||||
<div class="wi__c"><em>moss-tts.cpp</em> 48 kHz voice cloning <b>vs reference</b></div></a>
|
||||
<a class="wi rv" href="https://github.com/mudler/magpie-tts.cpp"><video src="/media/magpie.mp4" muted loop playsinline preload="none" data-lazy aria-label="magpie-tts.cpp synthesis benchmark"></video>
|
||||
<div class="wi__c"><em>magpie-tts.cpp</em> 9 languages, 5 voices <b>vs reference</b></div></a>
|
||||
</div>
|
||||
|
||||
<div class="reel rv" aria-hidden="true">
|
||||
<div class="reel__t">
|
||||
<span>vllm.cpp</span><span>parakeet.cpp</span><span>moss-transcribe.cpp</span><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>ced.cpp</span><span>voice-detect.cpp</span><span>voxtral-tts.c</span><span>vibevoice.cpp</span><span>rf-detr.cpp</span><span>locate-anything.cpp</span><span>depth-anything.cpp</span><span>face-detect.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span><span>privacy-filter.cpp</span><span>LocalVQE</span><span>local-store</span><span>apex-quant</span>
|
||||
<span>vllm.cpp</span><span>parakeet.cpp</span><span>moss-transcribe.cpp</span><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>ced.cpp</span><span>voice-detect.cpp</span><span>voxtral-tts.c</span><span>vibevoice.cpp</span><span>rf-detr.cpp</span><span>locate-anything.cpp</span><span>depth-anything.cpp</span><span>face-detect.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span><span>privacy-filter.cpp</span><span>LocalVQE</span><span>local-store</span><span>apex-quant</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="quiet rv">
|
||||
<div>
|
||||
<p class="quiet__tag">In development, not announced yet</p>
|
||||
<h4>vllm.cpp</h4>
|
||||
<p>vLLM ported to C++20, with paged attention, continuous batching and prefix caching, on CPU, CUDA, Metal and Vulkan. It installs as 66 MB instead of a 9.1 GB virtualenv, and it stays ahead of vLLM at every concurrency level we have measured so far. Still being finished, so treat the numbers as provisional.</p>
|
||||
</div>
|
||||
<video src="/media/vllm-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="vllm.cpp ahead of vLLM at every concurrency level"></video>
|
||||
</div>
|
||||
|
||||
<p class="rv mt2"><a class="btn btn--o" href="/engines/">All {{ len .Site.Data.engines.engines }} engines →</a></p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<!-- APEX -->
|
||||
<section class="paper" id="apex">
|
||||
<div class="shell">
|
||||
@@ -409,24 +336,6 @@
|
||||
<span>Deutsch</span><span>Español</span><span>français</span><span>日本語</span>
|
||||
<span>한국어</span><span>Português</span><span>Русский</span><span>中文</span>
|
||||
</div>
|
||||
{{/* The strongest sentence in this section is somebody else's, so it opens
|
||||
the section rather than closing it. Everything below is supporting
|
||||
evidence for what these three people already said. */}}
|
||||
<p class="kicker rv mt3">What other people say</p>
|
||||
<div class="headline">
|
||||
<a class="hq rv" href="https://x.com/ggerganov/status/2065447087311917459">
|
||||
<blockquote>“Some cool ggml-based work by @mudler_it recently, make sure to check it out.”</blockquote>
|
||||
<p class="hq__w">@ggerganov<b>Georgi Gerganov, author of llama.cpp and ggml</b></p>
|
||||
<span class="hq__go">On X, 2026 ↗</span></a>
|
||||
<a class="hq rv" href="https://x.com/badlogicgames/status/2061201400059531729">
|
||||
<blockquote>“What a wonderful project: parakeet.cpp. A ggml based parakeet inference pipeline that is 2x faster than my ONNX parakeet pipeline on Apple Silicon.”</blockquote>
|
||||
<p class="hq__w">@badlogicgames<b>Mario Zechner, author of pi.agent</b></p>
|
||||
<span class="hq__go">On X, 2026 ↗</span></a>
|
||||
<a class="hq rv" href="https://www.linkedin.com/posts/adimargolink_spinoza-saw-all-things-straining-to-become-activity-7468135820076634113-u3Hc">
|
||||
<blockquote>“Build something good enough that the community chooses to carry it beyond your reach. This week, Ettore Di Giacinto brought NVIDIA Parakeet to the CPU.”</blockquote>
|
||||
<p class="hq__w">Adi Margolin<b>On LinkedIn</b></p>
|
||||
<span class="hq__go">Read the post ↗</span></a>
|
||||
</div>
|
||||
|
||||
{{/* Names run as a sentence rather than a grid of pills. A pill wall of
|
||||
employers reads as a customer logo wall, which is a claim we are not
|
||||
@@ -535,7 +444,7 @@
|
||||
<h2 class="rv mt1" style="max-width:14ch">Running in about a minute.</h2>
|
||||
<p class="lede rv mt2">A container on any platform, a DMG on macOS, a binary on Linux, or a chart on Kubernetes. Backends download themselves the first time a model asks for one, so the base install stays small.</p>
|
||||
<div class="acts rv">
|
||||
<a class="btn" href="/docs/getting-started/">Installation guide →</a>
|
||||
<a class="btn" href="/docs/installation/index.html">Installation guide →</a>
|
||||
<a class="btn btn--o" href="https://discord.gg/uJAeKSAGDy">Join the Discord</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -67,7 +67,7 @@ h1,h2,h3{margin:0;font-family:var(--display),system-ui,sans-serif;font-weight:70
|
||||
section{position:relative;padding:var(--band) 0}
|
||||
.navy{background:rgba(4,19,28,var(--veil))} .surf{background:var(--navy1)} .deepbg{background:var(--deep)}
|
||||
.navy h1,.navy h2,.navy h3,.navy .lede,.navy p,.navy .kicker{text-shadow:var(--glowtext)}
|
||||
.navy .mono,.navy .chips span,.navy .apis span,.navy .sn__e span{text-shadow:0 1px 10px rgba(4,19,28,.85)}
|
||||
.navy .mono,.navy .chips span,.navy .apis span,.navy .sn__e a{text-shadow:0 1px 10px rgba(4,19,28,.85)}
|
||||
#field{position:fixed;inset:0;z-index:0;pointer-events:none;opacity:1;transition:opacity .7s ease}
|
||||
.top,main,footer{position:relative;z-index:1}
|
||||
.paper{background:var(--paper);color:var(--p-ink)}
|
||||
@@ -192,9 +192,9 @@ h2{font-size:clamp(2.1rem,5vw,3.9rem)}
|
||||
.sn__v u{text-decoration:none;color:var(--faint)}
|
||||
.sn p{color:var(--dim);font-size:.96rem;max-width:54ch}
|
||||
.sn__e{display:flex;flex-wrap:wrap;gap:.35rem}
|
||||
.sn__e span{font-family:'Geist Mono',monospace;font-size:.63rem;color:var(--dim);
|
||||
.sn__e a{font-family:'Geist Mono',monospace;font-size:.63rem;color:var(--dim);
|
||||
border:1px solid var(--line);padding:.24rem .5rem;border-radius:999px}
|
||||
.sn:hover .sn__e span{border-color:var(--cyan);color:var(--ink)}
|
||||
.sn:hover .sn__e a{border-color:var(--cyan);color:var(--ink)}
|
||||
|
||||
/* made with */
|
||||
.made{display:grid;gap:clamp(2rem,4vw,3.2rem);align-items:center;margin-top:2.6rem}
|
||||
@@ -396,6 +396,7 @@ tbody tr:hover td{background:rgba(95,205,228,.10)}
|
||||
.term__b{display:flex;gap:.5rem;align-items:center;padding:.6rem .85rem;border-bottom:1px solid var(--line2);
|
||||
font-family:'Geist Mono',monospace;font-size:.63rem;color:var(--faint)}
|
||||
.term pre{margin:0;padding:1.2rem;overflow-x:auto;font-family:'Geist Mono',monospace;font-size:.83rem;line-height:1.95}
|
||||
.pane{min-height:17rem}
|
||||
.term .p{color:var(--cyan)} .term .c{color:var(--cyan-hi)} .term .o{color:var(--dim)} .term .h{color:var(--ink)}
|
||||
.cpy{margin-left:auto;font-family:'Geist Mono',monospace;font-size:.63rem;color:var(--cyan);
|
||||
background:none;border:0;cursor:pointer;padding:0}
|
||||
|
||||
Reference in new issue
Block a user