mirror of
https://github.com/rclone/rclone.git
synced 2026-09-12 22:24:25 -04:00
Nick Craig-Wood
bd217a1082
Updating a file below --hidrive-upload-cutoff copied the whole file into an append-grown slice so the request could be retried, costing roughly twice the file size in transient allocations for every such update. Buffer the file in a multipart.NewRW from rclone's global page pool instead and return it to the pool once the request has finished. The pool.RW is seekable so retries re-send the same buffer. Accounting is applied as the buffer is sent so bandwidth limits and progress still track the upload. The request now carries an explicit Content-Length rather than being sent chunked. The upload is bounded by the size the source declares - a source that delivers more bytes than its declared size has the excess ignored, where previously the stream was sent to EOF. The FsPutRetry integration test covers the retry of a failed upload request and checks the buffers are returned to the pool.