mirror of
https://github.com/ollama/ollama.git
synced 2026-07-30 17:38:02 -04:00
d00622060f
Speculation used a parallel hierarchy of wrapper cache types that shadowed the live caches and reconciled against them on commit. Replace it with snapshot/restore on the live caches themselves: a cache snapshots itself as a write crosses each offset, and the runner commits a batched draft by restoring to the accepted count. The wrappers and the comparison plumbing around them are gone. Snapshots are lazy. A KV or rotating capture indexes into the live buffer and owns no memory until a destructive write forces a copy-out, so rejecting a draft is free. Recurrent layers now validate in the same batched pass rather than falling back to serial. A gated-delta layer reports its interior split offsets and hands back the recurrent state at each one, which the cache records as a snapshot.