mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 18:09:05 -04:00
c6c347ce13
GenerateImage hardcoded TilingParamsSetEnabled(vaep, false), so tiled VAE
decoding was unreachable from a model config even though all four upstream
setters were already bound in main.go.
Sampling runs in latent space, but the final VAE decode expands to full
resolution and needs one large compute buffer. At 1024x1024 that buffer
exceeds 8GB, which fails on two kinds of device: cards without the VRAM
for a full-frame decode, and drivers that cap a single allocation
regardless of how much memory is free. Mesa RADV reports a 4GiB
maxMemoryAllocationSize, so a Radeon 8060S with 74GiB of device-local
heap still cannot serve that decode:
[INFO ] sampling completed, taking 251.82s
[INFO ] decoding 1 latents
ggml_vulkan: Requested buffer size exceeds device buffer size limit:
ErrorOutOfDeviceMemory
[ERROR] vae: failed to allocate the compute buffer
[ERROR] decode_first_stage failed for latent 1
Every sampling step completes and then the run is discarded at the last
stage, so the whole generation is wasted.
Add three options, parsed in Load and applied per generation:
vae_tiling:true enable tiled decoding (bare flag also works)
vae_tile_size:512 tile size, or 512x384 for a rectangle
vae_tile_overlap:0.25 overlap between tiles
Tiling stays off unless requested, so existing models are unaffected. Tile
size and overlap only reach the library when the operator set them, which
keeps upstream's defaults rather than pushing a zero, and an unparseable
value is treated as absent for the same reason.
Truthy spellings match what load_model already accepts for its own bool
options, and the bare-flag form matches diffusion_model, so no new
convention is introduced.
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>