mirror of
https://github.com/rclone/rclone.git
synced 2026-11-08 02:32:57 -05:00
Nick Craig-Wood
59ea591bcb
The s3 and http backends each carried their own copy of the same
CheckRedirect policy. This moves it into lib/rest as
StripHeadersOnCrossHostRedirectFn, so that any backend which sends
credentials in headers to a host it does not control can use the same
policy. It logs at debug level when it strips headers so that a
resulting 401 from the redirect target can be diagnosed. A header
starting with "*" is matched without canonicalising, as that is how
SetHeader and Opts.ExtraHeaders send it.
This changes the s3 policy in two edge cases:
- It compared the raw URL host string, so a redirect to the same host
in different case or with an explicit default port stripped the
credentials. It now uses SameHost like the http backend, which treats
those as the same host.
- The downgrade check compared each hop with the previous one. It now
compares with the original request like the other backends, so a
chain which starts on plaintext HTTP may return to it.
The behaviour tests move to lib/rest. The backend tests which check the
policy is wired into each client are kept.
(cherry picked from commit cae10713e8)
Website | Documentation | Download | Contributing | Changelog | Installation | Forum
Rclone
Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.
Storage providers
- 1Fichier 📄
- Akamai Netstorage 📄
- Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
- Amazon S3 📄
- ArvanCloud Object Storage (AOS) 📄
- Bizfly Cloud Simple Storage 📄
- Backblaze B2 📄
- Box 📄
- Ceph 📄
- China Mobile Ecloud Elastic Object Storage (EOS) 📄
- Citrix ShareFile 📄
- Cloudflare R2 📄
- Cloudinary 📄
- Cubbit DS3 📄
- DigitalOcean Spaces 📄
- Digi Storage 📄
- Dreamhost 📄
- Drime 📄
- Dropbox 📄
- Enterprise File Fabric 📄
- Exaba 📄
- Fastly Object Storage 📄
- Fastmail Files 📄
- FileLu 📄
- Filen 📄
- Files.com 📄
- FlashBlade 📄
- FTP 📄
- GoFile 📄
- Google Cloud Storage 📄
- Google Drive 📄
- Google Photos 📄
- HDFS (Hadoop Distributed Filesystem) 📄
- Hetzner Object Storage 📄
- Hetzner Storage Box 📄
- HiDrive 📄
- Hitachi Content Platform (HCP) 📄
- HTTP 📄
- Huawei Cloud Object Storage Service(OBS) 📄
- Huawei Drive 📄
- iCloud Drive 📄
- ImageKit 📄
- Internet Archive 📄
- Internxt 📄
- Jottacloud 📄
- IBM COS S3 📄
- Impossible Cloud 📄
- Intercolo Object Storage 📄
- IONOS Cloud 📄
- Koofr 📄
- Leviia Object Storage 📄
- Liara Object Storage 📄
- Linkbox 📄
- Linode Object Storage 📄
- Magalu Object Storage 📄
- Mail.ru Cloud 📄
- Memset Memstore 📄
- MEGA 📄
- MEGA S4 Object Storage 📄
- Memory 📄
- Microsoft Azure Blob Storage 📄
- Microsoft Azure Files Storage 📄
- Microsoft OneDrive 📄
- Minio 📄
- Nextcloud 📄
- Blomp Cloud Storage 📄
- OpenDrive 📄
- OpenStack Swift 📄
- Oracle Cloud Storage 📄
- Oracle Object Storage 📄
- Outscale 📄
- OVHcloud Object Storage (Swift) 📄
- OVHcloud Object Storage (S3-compatible) 📄
- ownCloud 📄
- pCloud 📄
- Petabox 📄
- PikPak 📄
- Pixeldrain 📄
- premiumize.me 📄
- put.io 📄
- Proton Drive 📄
- QingStor 📄
- Qiniu Cloud Object Storage (Kodo) 📄
- Rabata Cloud Storage 📄
- Quatrix 📄
- Rackspace Cloud Files 📄
- RackCorp Object Storage 📄
- rsync.net 📄
- Scaleway 📄
- Scality (RING / ARTESCA) 📄
- Seafile 📄
- Seagate Lyve Cloud 📄
- SeaweedFS 📄
- Selectel Object Storage 📄
- Servercore Object Storage 📄
- SFTP 📄
- Shade 📄
- SMB / CIFS 📄
- Spectra Logic 📄
- Storj 📄
- SugarSync 📄
- Synology C2 Object Storage 📄
- Tencent Cloud Object Storage (COS) 📄
- Uloz.to 📄
- US3 Object Storage 📄
- Wasabi 📄
- WebDAV 📄
- Yandex Disk 📄
- Zadara Object Storage 📄
- Zero Services (ZERO-Z3) 📄
- Zoho WorkDrive 📄
- Zata.ai 📄
- The local filesystem 📄
Please see the full list of all storage providers and their features
Virtual storage providers
These backends adapt or modify other storage providers
- Alias: rename existing remotes 📄
- Archive: read archive files 📄
- Cache: cache remotes (DEPRECATED) 📄
- Chunker: split large files 📄
- Combine: combine multiple remotes into a directory tree 📄
- Compress: compress files 📄
- Crypt: encrypt files 📄
- Hasher: hash files 📄
- Union: join multiple remotes to work together 📄
Features
- MD5/SHA-1 hashes checked at all times for file integrity
- Timestamps preserved on files
- Partial syncs supported on a whole file basis
- Copy mode to just copy new/changed files
- Sync (one way) mode to make a directory identical
- Bisync (two way) to keep two directories in sync bidirectionally
- Check mode to check for file hash equality
- Can sync to and from network, e.g. two different cloud accounts
- Optional large file chunking (Chunker)
- Optional transparent compression (Compress)
- Optional encryption (Crypt)
- Optional FUSE mount (rclone mount)
- Multi-threaded downloads to local disk
- Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA
Installation & documentation
Please see the rclone website for:
Downloads
License
This is free software under the terms of the MIT license (check the COPYING file included in this package).