Ettore Di Giacinto
1cdcaf0152
feat: migrate to echo and enable cancellation of non-streaming requests ( #7270 )
...
* WIP: migrate to echo
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* tests
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2025-11-14 22:57:53 +01:00
Ettore Di Giacinto
089efe05fd
feat(backends): add system backend, refactor ( #6059 )
...
- Add a system backend path
- Refactor and consolidate system information in system state
- Use system state in all the components to figure out the system paths
to used whenever needed
- Refactor BackendConfig -> ModelConfig. This was otherway misleading as
now we do have a backend configuration which is not the model config.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2025-08-14 19:38:26 +02:00
Dave
3cddf24747
feat: Centralized Request Processing middleware ( #3847 )
...
* squash past, centralize request middleware PR
Signed-off-by: Dave Lee <dave@gray101.com >
* migrate bruno request files to examples repo
Signed-off-by: Dave Lee <dave@gray101.com >
* fix
Signed-off-by: Dave Lee <dave@gray101.com >
* Update tests/e2e-aio/e2e_test.go
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
---------
Signed-off-by: Dave Lee <dave@gray101.com >
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2025-02-10 12:06:16 +01:00
Shraddha
03974a4dd4
feat: tokenization with llama.cpp ( #4724 )
...
feat: tokenization
Signed-off-by: shraddhazpy <shraddha@shraddhafive.in >
2025-02-02 17:39:43 +00:00
Shraddha
5488fc3bc1
feat: tokenization endpoint ( #3710 )
...
endpoint to access the tokenizer
Signed-off-by: shraddhazpy <shraddha@shraddhafive.in >
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
Co-authored-by: Dave <dave@gray101.com >
2024-10-02 08:56:18 +02:00
