36f20f72f8
A 70 GB video checkpoint (longcat-video-avatar-1.5) could not be loaded on a
distributed cluster. The request failed with HTTP 500 after 1499.98s - exactly
the 25m00s cold-load ceiling - while staging was demonstrably healthy: 26 of 57
files and 39 GB transferred at a sustained ~26 MB/s, zero errors, no stalls. It
was not wedged, it was killed by a timer.
ModelLoadCeilingFor covers node selection, backend install, file staging and the
remote LoadModel. Install and load carry their own budgets; staging was covered
only by a FIXED 5-minute margin. But staging time is bytes over bandwidth, not a
constant: 70 GB at 26 MB/s needs ~45m against a 25m ceiling, so the failure is
deterministic for any sufficiently large model rather than a flake. Simply
raising the constant moves the cliff to the next model size - the deployment
target here is checkpoints of 600 GB and beyond.
The ceiling's real purpose is that "a wedged worker can never pin the lock
indefinitely". Progress, not elapsed time, is what distinguishes a wedged worker
from a large one. The hold is now a deadline that extends whenever the transfer
reports bytes and expires a 5-minute stall window after they stop:
- A large model transferring fine continues, for hours if needed.
- A worker that died mid-transfer still fails within the stall window.
Progress is observed at byte level on the transfer itself, via the existing
staging progress callback. Per-file completion would be too coarse - a single
600 GB shard would be indistinguishable from a stall for hours. The observation
point is back-pressured by the socket, so it reflects the network rather than
local disk reads. Observation is coarsened to one timer touch per stall/20 so
the per-read callback stays cheap.
The base budget (unchanged, and still derived from the install and load
timeouts) continues to cover the steps that report no progress, so
LOCALAI_NATS_MODEL_LOAD_TIMEOUT keeps working exactly as before. An absolute
cap of 24h bounds the hold even while progress keeps arriving, so a peer
trickling bytes forever cannot pin the advisory lock; 600 GB at the measured
26 MB/s is ~6.5h, so the cap sits far above any legitimate transfer.
Also fixes the incoherent layering the same error exposed: the resumable upload
carried a 1h retry budget nested inside the 25m ceiling, so the inner budget was
unreachable and the message still blamed it ("failed after 1 attempts within
1h0m0s budget") while the 25m parent was the actual killer. The upload now
adopts the caller's deadline when there is one, and applies its fixed budget
only when nothing above bounded it - which also stops a fixed 1h from
reintroducing the size cliff under the now-extendable parent.
This is the successor to #10968, where a hardcoded 5-minute LoadModel gRPC
timeout was replaced by this derived ceiling. Fixing the inner timeout exposed
the outer ceiling as the new binding constraint.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]
LocalAI website
LocalAI documentation website
Requirement
In this project, the Docsy theme component is pulled in as a Hugo module, together with other module dependencies:
$ hugo mod graph
hugo: collected modules in 566 ms
hugo: collected modules in 578 ms
github.com/google/docsy-example github.com/google/docsy@v0.5.1-0.20221017155306-99eacb09ffb0
github.com/google/docsy-example github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1
github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1 github.com/twbs/bootstrap@v4.6.2+incompatible
github.com/google/docsy/dependencies@v0.5.1-0.20221014161617-be5da07ecff1 github.com/FortAwesome/Font-Awesome@v0.0.0-20220831210243-d3a7818c253f
If you want to do SCSS edits and want to publish these, you need to install PostCSS
npm install
Running the website locally
Building and running the site locally requires a recent extended version of Hugo.
You can find out more about how to install Hugo for your environment in our
Getting started guide.
Once you've made your working copy of the site repo, from the repo root folder, run:
hugo server
Running a container locally
You can run docsy-example inside a Docker
container, the container runs with a volume bound to the docsy-example
folder. This approach doesn't require you to install any dependencies other
than Docker Desktop on
Windows and Mac, and Docker Compose
on Linux.
-
Build the docker image
docker-compose build -
Run the built image
docker-compose upNOTE: You can run both commands at once with
docker-compose up --build. -
Verify that the service is working.
Open your web browser and type
http://localhost:1313in your navigation bar, This opens a local instance of the docsy-example homepage. You can now make changes to the docsy example and those changes will immediately show up in your browser after you save.
Cleanup
To stop Docker Compose, on your terminal window, press Ctrl + C.
To remove the produced images run:
docker-compose rm
For more information see the Docker Compose documentation.
Troubleshooting
As you run the website locally, you may run into the following error:
➜ hugo server
INFO 2021/01/21 21:07:55 Using config file:
Building sites … INFO 2021/01/21 21:07:55 syncing static files to /
Built in 288 ms
Error: Error building site: TOCSS: failed to transform "scss/main.scss" (text/x-scss): resource "scss/scss/main.scss_9fadf33d895a46083cdd64396b57ef68" not found in file cache
This error occurs if you have not installed the extended version of Hugo. See this section of the user guide for instructions on how to install Hugo.
Or you may encounter the following error:
➜ hugo server
Error: failed to download modules: binary with name "go" not found
This error occurs if you have not installed the go programming language on your system.
See this section of the user guide for instructions on how to install go.