Jeffrey Morgan a1ca428c90 glm4moelite: fix attention scale calculation (#13893) ...

Use the original key dimension (qkNopeHeadDim + qkRopeHeadDim = 256) for
the attention scale instead of the MLA absorbed dimension (kvLoraRank +
qkRopeHeadDim = 576).

MLA absorption is a mathematically equivalent reorganization of the
attention computation - it should not change the effective attention
scale. The scale should match training, which uses 1/sqrt(256).

This improves tool calling and model looping issues.

2026-01-24 17:48:09 -08:00

..

imageproc

deepseekocr

2025-11-18 16:11:37 -08:00

input

batch: use tensors for outputs (#12185)

2025-09-15 14:33:06 -07:00

models

glm4moelite: fix attention scale calculation (#13893)

2026-01-24 17:48:09 -08:00

parsers

model: add lfm2 architecture and LFM2.5-1.2B-Thinking support (#13792)

2026-01-20 12:20:53 -08:00

renderers

model: add lfm2 architecture and LFM2.5-1.2B-Thinking support (#13792)

2026-01-20 12:20:53 -08:00

testdata

gemma2 impl

2025-03-11 14:35:08 -07:00

bytepairencoding_test.go

refactor: using testing.B.Loop

2025-10-10 13:25:29 -07:00

bytepairencoding.go

remove unnecessary code (#13502)

2025-12-16 15:11:26 -08:00

model_test.go

fix: leaf alt name (#12390)

2025-09-23 17:50:53 -07:00

model.go

Re-apply "model: add MLA absorption for glm4moelite" with fix (#13870)

2026-01-23 18:40:28 -08:00

sentencepiece_test.go

model: implement bert in ollama engine (#9080)

2025-09-15 15:35:59 -07:00

sentencepiece.go

fix(tokenizer): add special tokens to empty inputs (#13091)

2025-11-18 11:16:56 -08:00

textprocessor.go

model: handle multiple eos tokens (#10577)

2025-05-16 13:40:23 -07:00

vocabulary_test.go

fix(tokenizer): add special tokens to empty inputs (#13091)

2025-11-18 11:16:56 -08:00

vocabulary.go

fix(tokenizer): add special tokens to empty inputs (#13091)

2025-11-18 11:16:56 -08:00

wordpiece_test.go

nomic-embed-text model implementation (#13071)

2025-11-18 18:28:10 -08:00

wordpiece.go

nomic-embed-text model implementation (#13071)

2025-11-18 18:28:10 -08:00