Compare commits

..
Author SHA1 Message Date
localai-org-maint-bot 10d1e3e5cb fix(compose): request NVIDIA compute capability
The legacy NVIDIA device reservation requests utility without compute.
Docker derives driver capabilities from that list, leaving CUDA libraries
unavailable even when monitoring works.

Include compute in the legacy example and clarify the matching docs.

Assisted-by: Codex:GPT-6
2026-09-12 10:02:44 +00:00
6 changed files with 12 additions and 53 deletions

No files matched your search

@@ -135,13 +135,6 @@ build_tts_request(const backend::TTSRequest &request,
task.options["language"] = request.language();
}
// Saved voice profiles send ref_text; Fish Audio reads reference_text.
// Derive the alias before copying params so an explicit canonical key wins.
const auto reference_text = request.params().find("ref_text");
if (reference_text != request.params().end()) {
task.options["reference_text"] = reference_text->second;
}
// LAST, so an explicit params entry wins over anything derived above. That
// matters for "caption": a caller who sets params[caption] has named the
// exact string they want, and it must not be overwritten by `instructions`.
@@ -375,40 +375,6 @@ static void test_tts_language_and_params() {
"tts params: an explicit param overrides the derived caption");
}
static void test_tts_reference_transcript() {
backend::TTSRequest request;
request.set_text("New speech to generate.");
request.set_voice("reference.wav");
(*request.mutable_params())["ref_text"] = "The saved voice transcript.";
const auto task = build_tts_request(request, clip(24000, 1));
check(option_or(task.options, "reference_text", "") ==
"The saved voice transcript.",
"tts reference: saved transcript reaches Fish Audio's option");
check(option_or(task.options, "ref_text", "") ==
"The saved voice transcript.",
"tts reference: original transcript parameter is preserved");
check(task.text_input->text == "New speech to generate.",
"tts reference: transcript does not replace synthesis text");
(*request.mutable_params())["reference_text"] = "Explicit transcript.";
const auto explicit_task = build_tts_request(request, clip(24000, 1));
check(option_or(explicit_task.options, "reference_text", "") ==
"Explicit transcript.",
"tts reference: explicit canonical parameter wins over alias");
(*request.mutable_params())["reference_text"] = "";
const auto empty_task = build_tts_request(request, clip(24000, 1));
check(has_key(empty_task.options, "reference_text") &&
empty_task.options.at("reference_text").empty(),
"tts reference: explicit empty canonical parameter is preserved");
request.mutable_params()->clear();
const auto missing_task = build_tts_request(request, clip(24000, 1));
check(!has_key(missing_task.options, "reference_text"),
"tts reference: no transcript is invented when none was supplied");
}
static void test_sound_generation_minimal() {
backend::SoundGenerationRequest request;
request.set_text("a distant thunderstorm");
@@ -596,7 +562,6 @@ int main() {
test_tts_empty_language_is_not_a_language();
test_tts_clip_and_instructions();
test_tts_language_and_params();
test_tts_reference_transcript();
test_sound_generation_minimal();
test_sound_generation_full();
test_transform_text_absent();
+2 -1
View File
@@ -59,6 +59,7 @@ services:
# capabilities: [gpu, utility]
#
# For legacy NVIDIA driver (for older NVIDIA Container Toolkit):
# Request compute for CUDA libraries (libcuda.so.1) and utility for NVML.
# environment:
# NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
# init: true
@@ -68,7 +69,7 @@ services:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu, utility]
# capabilities: [gpu, compute, utility]
## Uncomment for PostgreSQL-backed knowledge base (see Agents docs)
# postgres:
-6
View File
@@ -217,12 +217,6 @@ voice conversion from the same weights.
## Family notes
- **Fish Audio voice cloning**: save a reference clip with its transcript in the
Voice Library, then select **Use in Text to Speech**. The backend accepts
`params.ref_text` as an alias for `params.reference_text` in both ordinary and
streaming speech requests. If you supply both parameters, `reference_text`
takes precedence. For direct requests with a reference file in `voice`, supply
its transcript in one of these parameters.
- **Supertonic**: use the `orig` GGUF package, whose weights are f32. The f16 package was
observed to reach `ggml_concat` with mismatched operand types and take the backend
process down with `SIGABRT` on the first request, rather than returning an error.
+6 -2
View File
@@ -417,8 +417,12 @@ usage is reported back to the frontend:
NVML library (and therefore `nvidia-smi`) is not available inside the
container. CUDA compute still works, but the worker cannot query free VRAM
and the Nodes page will show the node as fully used. Set
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` (or, with the NVIDIA CDI
runtime, list `capabilities: [gpu, utility]` on the device reservation).
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` when using the NVIDIA runtime.
For Docker Compose with `driver: nvidia`, use
`capabilities: [gpu, compute, utility]` on the device reservation.
Docker derives driver capabilities from this reservation, so include `compute`
for CUDA libraries such as `libcuda.so.1`. The `utility` capability alone
enables monitoring but does not provide CUDA libraries.
- **Run the container with `init: true` (or `docker run --init`).** The
worker process becomes PID 1 in the container and cannot reap zombies on
+4 -2
View File
@@ -88,8 +88,10 @@ page in the frontend shows the node as fully used, check two things:
NVML work inside the container. With `--gpus all` alone (or
`--runtime nvidia` without extra flags) only `compute` is wired in on
some driver versions. Add `-e NVIDIA_DRIVER_CAPABILITIES=compute,utility`
to your `docker run`, or `capabilities: [gpu, utility]` in compose /
Kubernetes device reservations.
to your `docker run`. For Docker Compose with `driver: nvidia`, use
`capabilities: [gpu, compute, utility]` on the device reservation.
Include `compute` for CUDA libraries such as `libcuda.so.1`; `utility`
alone only provides monitoring libraries and tools.
2. Pass `--init` to `docker run` (or `init: true` in compose) so the
container has a proper PID 1 reaper - otherwise short-lived child
processes like `nvidia-smi` can intermittently fail with