mirror of
https://github.com/rclone/rclone.git
synced 2026-09-13 14:49:29 -04:00
Nick Craig-Wood
cdabcc7cc4
Each upload allocated a fresh 48 MiB chunk buffer, so bulk transfers of many files churned allocations and GC, and small files paid for the full buffer. Buffer chunks with multipart.NewRW instead so chunk memory is reused across uploads and is part of rclone's central memory management. The pool.RW implements io.Closer, so the PATCH request body is wrapped in readers.NoCloser to stop the http transport closing it after a failed attempt and freeing its pages before the retry. The retry closure now seeks the chunk back to the start explicitly before resending, a short read of the source is reported as an error rather than sent as an under-length chunk, and the request carries an explicit ContentLength. The FsPutRetry integration test covers the retry of a failed upload request and checks the buffers are returned to the pool.