mirror of
https://github.com/rclone/rclone.git
synced 2026-03-05 15:50:57 -05:00
19a4d74ee7
When a part upload request fails error is returned and gCtx is cancelled. This does not prevent from other parts being tried. They immediately fail due to a canceled context, but are retried by rclone anyway... Example AWS debug output ``` ----------------------------------------------------- 2020/02/11 14:12:17 DEBUG: Retrying Request s3/UploadPart, attempt 4 2020/02/11 14:12:17 DEBUG: Request s3/UploadPart Details: ---[ REQUEST POST-SIGN ]----------------------------- PUT /backuptest-rclone/huge/file.db?partNumber=11&uploadId=190939b4-3c43-4b98-ac11-92303e3f11b0 HTTP/1.1 Host: 192.168.100.99:9000 User-Agent: aws-sdk-go/1.23.8 (go1.13.1; linux; amd64) Content-Length: 5242880 Authorization: AWS4-HMAC-SHA256 Credential=miniouser/20200211/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-md5;expect;host;x-amz-content-sha256;x-amz-date, Signature=3fc03a01f651cec09b05290459e9ceb26db9a8aa00c4e1b16e8cf5617eb81da8 Content-Md5: XzY+DlipXwbL6bvGYsXftg== Expect: 100-Continue X-Amz-Content-Sha256: c036cbb7553a909f8b8877d4461924307f27ecb66cff928eeeafd569c3887e29 X-Amz-Date: 20200211T131217Z Accept-Encoding: gzip ----------------------------------------------------- http://192.168.100.99:9000/backuptest-rclone/huge/file.db?partNumber=11&uploadId=190939b4-3c43-4b98-ac11-92303e3f11b0 2020/02/11 14:12:17 DEBUG: Response s3/UploadPart Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 500 InternalServerError Content-Length: 0 ----------------------------------------------------- UploadPartWithContext() error InternalError: We encountered an internal error. Please try again status code: 500, request id: , host id: 2020/02/11 14:12:18 DEBUG ERROR: Request s3/UploadPart: ---[ REQUEST DUMP ERROR ]----------------------------- context canceled ------------------------------------------------------ UploadPartWithContext() error RequestCanceled: request context canceled caused by: context canceled 2020/02/11 14:12:20 DEBUG ERROR: Request s3/UploadPart: ---[ REQUEST DUMP ERROR ]----------------------------- context canceled ------------------------------------------------------ UploadPartWithContext() error RequestCanceled: request context canceled caused by: context canceled 2020/02/11 14:12:22 DEBUG ERROR: Request s3/UploadPart: ---[ REQUEST DUMP ERROR ]----------------------------- context canceled ------------------------------------------------------ UploadPartWithContext() error RequestCanceled: request context canceled caused by: context canceled ``` This adds a fail fast behaviour in case the context was cancelled.
Website | Documentation | Download | Contributing | Changelog | Installation | Forum
Rclone
Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers.
Storage providers
- 1Fichier 📄
- Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
- Amazon Drive 📄 (See note)
- Amazon S3 📄
- Backblaze B2 📄
- Box 📄
- Ceph 📄
- Citrix ShareFile 📄
- DigitalOcean Spaces 📄
- Dreamhost 📄
- Dropbox 📄
- FTP 📄
- GetSky 📄
- Google Cloud Storage 📄
- Google Drive 📄
- Google Photos 📄
- HTTP 📄
- Hubic 📄
- Jottacloud 📄
- IBM COS S3 📄
- Koofr 📄
- Mail.ru Cloud 📄
- Memset Memstore 📄
- Mega 📄
- Memory 📄
- Microsoft Azure Blob Storage 📄
- Microsoft OneDrive 📄
- Minio 📄
- Nextcloud 📄
- OVH 📄
- OpenDrive 📄
- OpenStack Swift 📄
- Oracle Cloud Storage 📄
- ownCloud 📄
- pCloud 📄
- premiumize.me 📄
- put.io 📄
- QingStor 📄
- Rackspace Cloud Files 📄
- Scaleway 📄
- SFTP 📄
- StackPath 📄
- SugarSync 📄
- Wasabi 📄
- WebDAV 📄
- Yandex Disk 📄
- The local filesystem 📄
Please see the full list of all storage providers and their features
Features
- MD5/SHA-1 hashes checked at all times for file integrity
- Timestamps preserved on files
- Partial syncs supported on a whole file basis
- Copy mode to just copy new/changed files
- Sync (one way) mode to make a directory identical
- Check mode to check for file hash equality
- Can sync to and from network, e.g. two different cloud accounts
- Optional large file chunking (Chunker)
- Optional encryption (Crypt)
- Optional cache (Cache)
- Optional FUSE mount (rclone mount)
- Multi-threaded downloads to local disk
- Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna
Installation & documentation
Please see the rclone website for:
Downloads
License
This is free software under the terms of MIT the license (check the COPYING file included in this package).