Aaron Pham 3b2ac1cd59 feat: support continuous batching on generate (#375) ...

* feat: support continuous batching on `generate`

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add changelog

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

2023-09-19 03:04:59 -04:00

..

bundle

fix: set default serialisation methods (#355)

2023-09-18 02:26:53 -04:00

cli

fix: set default serialisation methods (#355)

2023-09-18 02:26:53 -04:00

models

fix(prompt): correct export extra objects items (#351)

2023-09-14 03:42:28 -04:00

playground

chore: ignore new lines split [skip ci]

2023-09-01 17:00:49 +00:00

serialisation

fix(serialisation): vLLM safetensors support (#324)

2023-09-12 17:44:01 -04:00

utils

fix(serialisation): vLLM safetensors support (#324)

2023-09-12 17:44:01 -04:00

__init__.py

chore: ignore new lines split [skip ci]

2023-09-01 17:00:49 +00:00

__main__.py

fix: persistent styling between ruff and yapf (#279)

2023-08-30 11:37:41 -04:00

_assign.py

feat: continuous batching with vLLM (#349)

2023-09-14 03:09:36 -04:00

_embeddings.py

chore: ignore new lines split [skip ci]

2023-09-01 17:00:49 +00:00

_generation.py

chore: ignore new lines split [skip ci]

2023-09-01 17:00:49 +00:00

_llm.py

feat: support continuous batching on generate (#375)

2023-09-19 03:04:59 -04:00

_quantisation.py

fix(gptq): use upstream integration (#297)

2023-09-04 14:05:50 -04:00

_service.py

feat: support continuous batching on generate (#375)

2023-09-19 03:04:59 -04:00

client.py

fix: persistent styling between ruff and yapf (#279)

2023-08-30 11:37:41 -04:00

exceptions.py

fix: persistent styling between ruff and yapf (#279)

2023-08-30 11:37:41 -04:00

py.typed

refactor: monorepo (#203)

2023-08-15 02:11:14 -04:00

testing.py

fix(gptq): use upstream integration (#297)

2023-09-04 14:05:50 -04:00