mirror of
https://github.com/rclone/rclone.git
synced 2026-09-14 07:14:20 -04:00
SillyZir
393544b116
Each resumable upload allocated a fresh chunk-sized buffer (8 MiB by default), so bulk transfers of many files churned allocations and GC. Buffer chunks with multipart.NewRW instead — the global page pool used by the other backends — so chunk memory is reused across uploads and bounded by rclone's central memory management. The pool.RW is seekable, which also keeps chunk reads repeatable for retries. The pool.RW implements io.Closer, so http.NewRequestWithContext upgraded it to the request body and the transport closed it after each attempt, returning its pages to the global pool — a chunk retried after a 5xx then read a freed buffer and panicked in pool.(*RW).readPage. Wrap the request body in readers.NoCloser so the transport can't take ownership and the upload loop remains solely responsible for the buffer's lifetime. Add a regression test that fails a chunk with a 500 and then accepts the retry; it reproduces the panic without the fix. Fixes #9684