mirror of
https://github.com/rclone/rclone.git
synced 2026-09-16 00:03:05 -04:00
Nick Craig-Wood
6613d4ec63
Each upload allocated a fresh chunk-sized buffer (10 MiB by default) regardless of the file size, so bulk transfers of many files churned allocations and GC. Buffer chunks with multipart.NewRW instead so chunk memory is reused across uploads and is part of rclone's central memory management. The pool.RW is seekable, so the existing rewind on retry carries over. The body goes through lib/rest which already wraps it so the transport can't close the pool buffer. The upload loop closes it after every chunk, on error paths included. A source which ends before the declared size is now reported as a short read before the chunk is sent rather than as an incomplete write afterwards. The FsPutRetry integration test covers the retry of a failed upload request and checks the buffers are returned to the pool.