mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-04 12:22:22 -04:00
* blog: rewrite the engines post without the AI tells
The HN thread on this post (item 49125065) spent most of its comments on the
writing rather than the engines. Readers quoted specific lines back as tells.
This is the same post with the same numbers, edited against the updated
no-ai-slop skill.
Every figure, table and link is unchanged, except that "27% of the memory"
is now the underlying 363 MB against 1328 MB from the table.
Two substantive framing fixes, both from the reply draft in
hn-reply-engines-post.md:
- vllm.cpp is no longer implied to be a speed win. The table is a tie, the
result is the install size, and the post now says so before a reader has to
work it out and post about it.
- Added one line on the language mix. Readers took the C++/Python/Go tree as
incoherence rather than as a Go core with per-ecosystem backends.
Cut throughout: the ledger metaphor ("what those ports buy", "not paid for in
throughput"), unearned framing ("the honest reading is", "has nothing to do
with"), the shape summary ("that is the general shape of these wins"),
confident deference ("people who are better at those models than we are"),
self-grading numbers ("a good result for a 66 MiB binary"), verbless
comparisons, three of the four exactness idioms, and the aphoristic headings
and verdicts. The double-tricolon summary is one plain clause now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* blog, website: same anti-slop sweep over the rest of the site
One-by-one pass over the other four posts and the site templates, with the
same rules used on the engines post. All figures, tables, links and PR
numbers are unchanged everywhere; the edits are to prose only.
apex-moe-quantization: ledger metaphors were the main issue, eight uses of
buy/cost/pay/spend for things that are not money. Also "the honest reading
is", "that is the comparison that matters", and two section-ending aphorisms
("Size is a speed knob as much as a memory knob", "Q6_K is the ceiling worth
paying for").
localai-since-march-2023: light touch, this one already reads like a person.
Removed "the curve is not the point", a "not the feature list, but the four
decisions" contrast, and two "X is what made / is the piece that" forms.
parakeet-cpp-asr-on-cpu: six exactness idioms across one post, "byte for
byte" twice, "character for character" twice, "byte-identical" twice and
"bit-identical" once, including in the title. Down to one, kept where the
precision is load-bearing. Also the "what end-of-utterance detection buys
you" heading and the "we say so rather than averaging it away" flex.
what-landed-in-localai-4-8: no changes. It is dense, flat and ends every
section on a PR number or a plain fact, which is the shape the other posts
should look like.
Site templates: "Most backends wrap somebody else's engine. These do not."
was the same contrast the engines post opened with. Also "Not a degraded mode
that technically runs", "A port only ships once it matches the original",
"Speed is the part we then go and win ... not a marketing run", and the last
"byte for byte" on the landing page.
Hugo builds clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* website: it is eighteen engines, not nineteen
Three places said nineteen: the /engines/ page description, the JUL 2026
timeline entry on the landing page, and the header comment in
data/engines.yaml.
Eighteen is right, confirmed two ways. The "Backends built by us" table in
the README has exactly 18 rows, and data/engines.yaml has 19 entries of which
one is apex-quant, which is a quantization recipe rather than an engine. The
two lists otherwise match name for name.
The yaml comment is the likely origin: it read "the nineteen native engines
the LocalAI team wrote, and the one quantization recipe that feeds them",
which counts apex-quant twice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
131 lines
5.7 KiB
HTML
131 lines
5.7 KiB
HTML
{{ define "main" }}
|
|
{{ $data := .Site.Data.engines }}
|
|
{{ $cats := $data.categories }}
|
|
{{ $engines := $data.engines }}
|
|
<main id="top">
|
|
|
|
<!-- HEAD -->
|
|
<section class="hero navy">
|
|
<div class="shell">
|
|
<p class="kicker fd" style="margin-top:0">Engines we build</p>
|
|
<h1 class="eng-h1"><u><b>Eighteen engines,</b></u><u><b><s>written from scratch.</s></b></u></h1>
|
|
<div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
|
|
<p class="lede fd mt2">Most LocalAI backends wrap somebody else's engine. These were written from scratch, each one because the thing we needed was a multi-gigabyte Python install, or closed, or nobody had built it yet. What ships instead is a binary and a GGUF file, checked against the reference implementation in CI, running on the machine you already own.</p>
|
|
<div class="acts fd">
|
|
<a class="btn" href="/#start">Install LocalAI <span>→</span></a>
|
|
<a class="btn btn--o" href="/docs/features/backends/">How backends work</a>
|
|
</div>
|
|
<div class="figures fd">
|
|
<div><b class="tnum" data-count="{{ len $engines }}">0</b><span>Engines</span></div>
|
|
<div><b class="tnum" data-count="{{ len $cats }}">0</b><span>Modalities</span></div>
|
|
<div><b class="tnum" data-count="0">0</b><span>Python at inference</span></div>
|
|
<div><b class="tnum" data-count="0" data-text="MIT">0</b><span>Licence</span></div>
|
|
</div>
|
|
</div>
|
|
</section>
|
|
|
|
<!-- CATALOGUE -->
|
|
<section class="navy" id="catalogue">
|
|
<div class="shell">
|
|
|
|
<div class="engf rv" role="group" aria-label="Filter engines by modality">
|
|
<button type="button" class="engf__c" data-cat="all" aria-pressed="true">All <b>{{ len $engines }}</b></button>
|
|
{{ range $cats }}
|
|
{{ $n := len (where $engines "category" .id) }}
|
|
{{ if gt $n 0 }}<button type="button" class="engf__c" data-cat="{{ .id }}" aria-pressed="false">{{ .label }} <b>{{ $n }}</b></button>{{ end }}
|
|
{{ end }}
|
|
</div>
|
|
|
|
{{ range $cats }}
|
|
{{ $inCat := where $engines "category" .id }}
|
|
{{ if gt (len $inCat) 0 }}
|
|
<div class="engg" data-group="{{ .id }}" id="{{ .id }}">
|
|
<div class="engg__h rv">
|
|
<p class="kicker">{{ .label }}</p>
|
|
<p class="engg__b">{{ .blurb }}</p>
|
|
</div>
|
|
<div class="engs">
|
|
{{ range $inCat }}
|
|
{{ $e := . }}
|
|
<a class="eng rv{{ if $e.featured }} eng--f{{ end }}" data-cat="{{ $e.category }}" href="{{ $e.repo }}">
|
|
{{ with $e.media }}
|
|
<div class="eng__m">
|
|
<video src="{{ . }}"{{ with $e.poster }} poster="{{ . }}"{{ end }} muted loop playsinline preload="none" data-lazy aria-label="{{ $e.name }}: {{ $e.tagline }}"></video>
|
|
</div>
|
|
{{ end }}
|
|
<div class="eng__b">
|
|
<p class="eng__k"><span>{{ $e.language }}</span>{{ with $e.status }}<b class="eng__s">{{ . }}</b>{{ end }}</p>
|
|
<h3 class="eng__n">{{ $e.name }}</h3>
|
|
<p class="eng__t">{{ $e.tagline }}</p>
|
|
{{ with $e.highlights }}
|
|
<ul class="eng__l">{{ range . }}<li>{{ . }}</li>{{ end }}</ul>
|
|
{{ end }}
|
|
{{ with $e.clips }}
|
|
<div class="eng__x">
|
|
{{ range . }}
|
|
<figure>
|
|
<video src="{{ .src }}" muted loop playsinline preload="none" data-lazy aria-label="{{ .caption }}"></video>
|
|
<figcaption>{{ .caption }}</figcaption>
|
|
</figure>
|
|
{{ end }}
|
|
</div>
|
|
{{ end }}
|
|
<span class="eng__go">{{ $e.name }} on GitHub ↗</span>
|
|
</div>
|
|
</a>
|
|
{{ end }}
|
|
</div>
|
|
</div>
|
|
{{ end }}
|
|
{{ end }}
|
|
|
|
<p class="engn rv" role="status" hidden>Nothing in that modality yet.</p>
|
|
</div>
|
|
</section>
|
|
|
|
<!-- CLOSE -->
|
|
<section class="deepbg" id="why">
|
|
<div class="shell">
|
|
<div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
|
|
<p class="kicker rv">The rule we hold them to</p>
|
|
<h2 class="rv mt1" style="max-width:20ch">We do not ship a port until it matches the original.</h2>
|
|
<p class="lede rv mt2">Every engine here is gated against the framework it replaces, on the same input, on the same machine. That means a transcript identical to the reference, boxes that land on the same pixels, or a waveform inside a stated tolerance. Speed work comes after that, and the numbers on this page come out of each engine's own benchmark suite.</p>
|
|
<div class="acts rv">
|
|
<a class="btn" href="/#start">Install LocalAI →</a>
|
|
<a class="btn btn--o" href="https://github.com/mudler/LocalAI">LocalAI on GitHub ↗</a>
|
|
</div>
|
|
</div>
|
|
</section>
|
|
|
|
</main>
|
|
|
|
<script>
|
|
(function(){
|
|
var chips = [].slice.call(document.querySelectorAll('.engf__c'));
|
|
var cards = [].slice.call(document.querySelectorAll('.eng'));
|
|
var groups = [].slice.call(document.querySelectorAll('.engg'));
|
|
var none = document.querySelector('.engn');
|
|
if (!chips.length) return;
|
|
function apply(cat){
|
|
var shown = 0;
|
|
cards.forEach(function(c){
|
|
var on = (cat === 'all' || c.dataset.cat === cat);
|
|
c.hidden = !on;
|
|
if (on) shown++;
|
|
});
|
|
groups.forEach(function(g){
|
|
g.hidden = !(cat === 'all' || g.dataset.group === cat);
|
|
});
|
|
chips.forEach(function(c){ c.setAttribute('aria-pressed', String(c.dataset.cat === cat)); });
|
|
if (none) none.hidden = shown > 0;
|
|
try { history.replaceState(null, '', cat === 'all' ? location.pathname : location.pathname + '#' + cat); } catch (e) {}
|
|
}
|
|
chips.forEach(function(c){
|
|
c.addEventListener('click', function(){ apply(c.dataset.cat); });
|
|
});
|
|
var start = (location.hash || '').replace('#','');
|
|
if (start && chips.some(function(c){ return c.dataset.cat === start; })) apply(start);
|
|
})();
|
|
</script>
|
|
{{ end }}
|