After PUTting a file to the upload URL, Yandex keeps the file locked
for writing until the upload operation finishes committing on the
server. The PUT returned before this happened, so the following
SetModTime raced the still-in-progress write and got spurious 500
Internal Server Error responses.
Capture the operation_id returned with the upload URL and poll the
operation status until it reports success before returning, so the file
is fully committed before we access it.
The background chunk cleaner slept for the whole ChunkCleanInterval
(default 1 minute) before checking its stop channel, and only ran
CleanUpCache via the select default branch. This meant a cache that
had been stopped by StopBackgroundRunners could keep running
CleanUpCache for up to an interval afterwards.
The cache backend tests all share a single on-disk chunk store (the
TestInternalCache remote), so a lingering cleaner from a finished test
could call CleanChunksBySize and os.RemoveAll chunks that a later,
unrelated test had just written. The later test would then read a
chunk back and get an unexpected EOF - eg
TestInternalMaxChunkSizeRespected failing intermittently on CI.
Wait on a timer and the stop channel together so a stop is honoured
immediately and the cleaner can never run again once stopped.
A folder shortcut pointing at one of its own ancestor folders (for
example a shortcut to FolderX placed inside FolderX) made rclone recurse
forever when dereferencing shortcuts, duplicating the folder contents
until the disk was full.
Rclone now detects when a folder shortcut targets an ancestor directory
using the directory cache, leaves that shortcut out of the listing and
logs an ERROR, so the rest of the drive can still be copied.
Fixes#7118Fixes#9565Closes#9051
cleanRootPath used filepath.Abs which prepends the current directory,
but the resulting absolute path does not always refer to the same
directory as the original relative path - for example when the current
directory is shadowed by a mount or has been removed. This made
"rclone copy --links . ../dst" fail where "cp -ra . ../dst" succeeds.
rclone now cleans the path lexically on non-Windows platforms instead,
leaving relative roots relative so the OS resolves them against the live
working directory. Windows still makes the path absolute as required for
UNC long-path conversion.
When the stored token is expired and refreshOrReLogin fails, the
returned error wrapped the original 401 and discarded authErr, hiding
the actionable reason (e.g. "account requires 2FA - please run: rclone
config reconnect remote:"). Wrap both errors so the user sees why
re-auth failed and what to do about it.
Fixes#9583
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Zoho's WorkDrive listing API returns "401 R008 Unauthorized access"
(not a 404) when a folder id no longer resolves to a listable folder,
because it was deleted or never existed. A freshly refreshed token still
gets it, so it is not a token problem and retrying it is futile - and can
escalate to a 429 F7008 rate-limit penalty.
Handle it as a missing directory instead: shouldRetry no longer retries a
bare R008 401, listAll maps it to fs.ErrorDirNotFound, and
readMetaDataForPath flushes the stale parent from the dircache and reports
the object as not found so a later create re-resolves the parent. This lets
the VFS self-heal a stale cached directory id instead of hard-failing the
operation, and stops the VFS integration tests failing on a stale directory id.
Fixes#9578
The /stream/upload endpoint's overwrite flag is the x-prefixed header
x-override-name-exist, matching its siblings x-filename, x-parent_id and
x-streammode. uploadLargeFile sent the un-prefixed override-name-exist, which
the endpoint ignores, so overwriting a file of 10 MiB or larger created a
renamed duplicate instead of updating the existing file. The small upload API
passes the flag as a query parameter and is unaffected.
The folder list limiter granted the caller-supplied time immediately in
the burst phase, so concurrent callers observing time.Now() out of order
could record grant times that moved backwards. The sliding safety log
indexes grants as an ordered history, so out-of-order grants could also
breach the rolling-window cap. Clamp each grant to be at or after the
previous one so grant times are always monotonic.
Seafile returns a content-addressed id (a git-like hash of the file
contents) rather than a unique per-object identifier. Two different
files with identical contents are therefore given the same id, even on
different servers.
operations.SameObject assumes IDs uniquely identify an object, so when
syncing between Seafile remotes it treated two distinct but
identical-content files as the same object. The guard in NeedTransfer
if Equal(ctx, src, dst) && !SameObject(src, dst)
then evaluated to false for unchanged files (Equal was true because the
sizes matched, and SameObject was true because the ids matched), so
rclone re-uploaded them on every sync.
Seafile has no modtime and no usable content hash, so dropping the IDer
implementation makes SameObject fall back to path comparison and lets
unchanged files be skipped correctly.
Fixes#8300
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a range request Azure returns the whole-blob MD5 in the
x-ms-blob-content-md5 header (BlobContentMD5) and leaves Content-MD5
empty. The download metadata decoder only read Content-MD5, so every
ranged read overwrote the object's MD5 with an empty string.
With --vfs-cache-mode full this changed the object's fingerprint between
opens, so the VFS cache judged every reopened file as stale and
re-downloaded it, defeating the cache and inflating egress. Prefer
BlobContentMD5 and never overwrite a known hash with an empty one.
With -l/--links rclone recreates a .rclonelink object as a symlink. A
malicious or compromised source could serve a symlink whose target points
outside the destination, plus a sibling object whose path traverses it, so
that rclone followed the planted symlink and wrote outside the destination
causing arbitrary file write.
When translating symlinks, rclone now performs all destination writes
(directory creation, file writes and symlink creation) through an os.Root
anchored at the destination. os.Root resolves every path component relative
to the destination's file descriptor and refuses any that escapes the root,
even under concurrent modification, so a planted symlink can never be
traversed out of the destination.
Symlinks are still reproduced verbatim - including ones whose target points
outside the destination - so backups remain faithful. Only writing
*through* such a link is refused. In-tree symlinks are unaffected.
Fixes CVE-2026-54572
Fixes GHSA-cf44-9pgv-m4xc
When applying the "mode" from --metadata the local backend cast the
source value straight to an os.FileMode, so a source that supplied a
mode with Go's setuid, setgid or sticky bits set would have those bits
applied to the freshly written file. As both the file content and its
metadata come from the source remote, a malicious source could plant a
setuid binary, and a victim running "rclone copy -M" as root against
an untrusted remote could end up with a root-owned setuid binary with
attacker-controlled content.
Rclone records "mode" in the unix st_mode layout where the special
bits live in different positions to Go's os.FileMode, so honest
sources never actually round-tripped these bits in the first place.
Apply only the permission bits by default, which closes this off and
is backwards compatible, and add the --local-metadata-restore-special-bits
lag to restore the previous behaviour for trusted sources such as
restoring a system backup made by rclone.
See: GHSA-945v-v9p3-v5xw
The CheckRedirect policy strips the X-Amz-Security-Token header when a
redirect chain crosses a host, but it only compared the host and ignored
the scheme. A redirect that kept the same host:port but downgraded
https:// to http:// was treated as the same host, so the STS session
token was re-sent over a plaintext connection where it could be observed.
Fixes GHSA-cf44-9pgv-m4xc
When mounting or otherwise opening an S3 prefix without a trailing
slash, rclone probes the path with a HEAD request to see whether it is
actually a file. Since v1.72.0 (#8975) any error other than "not
found" from that probe was fatal, so credentials scoped to a prefix -
which return 403 rather than 404 for the prefix key - could no longer
open the prefix at all.
6440052fbd s3: fix single file copying behavior with low permission
A 403 on the probe is ambiguous: it can mean either "this is the file
you named but you may not HEAD it" or "this is a prefix you may list
but not HEAD". When the HEAD is not permitted we now fall back to a
listing to disambiguate: if the path has children it is treated as a
directory, otherwise it is treated as a file.
Fixes#9582
rclone's shared Google Drive and Google Photos client_id is being
retired and will stop working during 2026. When creating a new remote
that would use it, the config wizard now warns the user and asks the
user to enter their own client_id and secret instead. Service account
and environment auth are unaffected as they don't use the shared
client_id.
See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
The shared Google Drive and Google Photos client_id is being retired and
will stop working during 2026. Warn users who rely on it (ie who have not
configured their own client_id) so they can create their own in advance.
The warning is only shown for auth flows that actually use the shared
client_id, not for service account, environment or anonymous auth.
See: https://forum.rclone.org/t/google-drive-and-google-photos-users-action-required/54005
WorkDrive throttles its listing API (GET files/{id}/files) PER
folder, independently of the overall request rate: at most ~19
listings of one folder are allowed in any rolling ~60s window and
the 20th returns F7008 with a ~300s Retry-After (measured live -
every observed trip landed exactly on the 20th listing inside a
window). fstests re-lists the same working directory after almost
every sub-operation, which is why the integration suite could not
pass.
Add a per-folder listing limiter with a true per-window cap: each
window starts with --zoho-list-folder-burst listings passing
back-to-back (the burst re-arms at every window boundary, so a
sync re-listing one directory a few times never waits), the rest
of the budget is spaced evenly across the window, and a sliding
log of recent listings guarantees no rolling window ever exceeds
--zoho-list-folder-limit (default 19) per --zoho-list-folder-window
(default 60s) for any traffic pattern. The registry is
process-wide and keyed by region+folder id so every Fs instance
shares one budget per physical folder; idle entries are evicted
after a window, which is lossless because Zoho's window has also
cleared by then.
Defaults were validated against the live service: bursts of 4-6
under the 19-per-60s cap ran clean while an over-cap probe tripped
F7008 exactly at the 20th listing, and a full test_all -backends
zoho run passes cleanly.
Fixes#9570
A 429 stall was only visible as a DEBUG pacer line, so without -vv
rclone appeared to hang for 2-5 minutes. In one night's batch logs 17
job starts produced only 4 completions because the silent stalls
looked like hangs and the jobs kept getting killed, re-triggering the
throttle.
Log the first 429 of each throttle episode at NOTICE with the server
message and the wait time. An episode ends when a request succeeds
after the penalty window; retries within an episode stay at DEBUG via
the existing pacer logging. State is two atomics behind a pointer on
Fs, so shallow Fs copies share it and concurrent checkers are safe.
See #9570
Zoho throttling is account/plan-dependent; measurements show a
sustainable listing rate of ~6 requests/s on a production account -
going faster drains a token bucket and stalls ~2 minutes per
Retry-After, which is strictly slower overall. Add per-remote pacer
options (default 6/1) using the same token-bucket pacer as the Google
Drive backend. Set --zoho-tpslimit 0 to disable the cap.
See #9570
Zoho WorkDrive now sends a Retry-After header on 429 (it did not when
the backend was written). Waiting the hard-coded 60s retried too early
when the server asked for more (Retry-After: 299 is common) and the
penalty escalated (observed 84s -> 239s). Honour the header plus a 1s
margin - retrying at exactly Retry-After still finds an empty token
bucket and burns ~16 immediate 429s - and keep 60s as the fallback
when the header is absent.
shouldRetry becomes a method on *Fs so the retry decision has access
to the remote's state; later commits build on this.
See #9570
The mega backend keeps the whole account as an in-memory tree. go-mega's
Delete with destroy=true (hard_delete) removed the node from its lookup
table but left it in its parent's children, so a hard deleted file kept
showing up in directory listings until the tree was reloaded. In a
long-running mount or serve the file reappeared once the VFS directory
cache expired and re-read the backend.
Update go-mega to pick up the fix, and let the ghost test exercise
whichever delete mode the remote is configured with.
The mega backend keeps the whole account as an in-memory tree which
go-mega reconciles asynchronously from the server's event stream. After
an upload, delete or move the optimistic local update could be undone
moments later when go-mega replayed the corresponding server event,
re-adding a node to its parent. In a long-running process such as mount
or serve this could briefly show a just-deleted file in a listing, or
make a moved file reappear shortly afterwards.
Wait for the server to confirm uploads, deletes and moves so the
in-memory tree is settled before returning, matching what Rmdir and
Purge already do. The move wait was also previously started after the
server-side move so only the rename was covered.
Add an internal test reproducing the lingering listing via a tight
put/remove/list loop.
The HeadObject path translates a 404 (Not Found) and a 405 (Method Not
Allowed) response into fs.ErrorObjectNotFound, but the GetObject path
returns the raw AWS SDK error instead. With --s3-no-head-object set,
Object.Open() is the first request to hit the source for a GET
operation, so callers that rely on fs.ErrorObjectNotFound see an
opaque "operation error S3: GetObject, ... StatusCode: 404" string
when --s3-no-head-object is set.
Mirror the error handling of HeadObject. The 405 (Method Not Allowed)
case applies to GetObject for the same reason it applies to
HeadObject: A request that specifies the versionId of a delete marker
returns 405 instead of 404 (commit 8470bdf810, AWS S3 delete-marker
docs).
Co-authored-by: Jan Schlien <rclone-github@jan-o-sch.net>
This adds two new Dropbox backend flags:
--dropbox-skip-shared-folders skips all shared folder mount points
regardless of ownership.
--dropbox-skip-unowned-folders only skips shared folders that are
not owned by the current user.
These help avoid backing up the same shared folder multiple times when
backing up multiple Dropbox accounts.
Fixes#9514
This PR fixes a bug in the WebDAV backend where directories or files could
randomly "disappear" from listings due to strict and fragile multi-status code
parsing.
Co-authored-by: bright <nako_ruru@sina.com>
Add a redirect policy to the S3 HTTP client so X-Amz-Security-Token is
removed once a redirect chain crosses hosts. Keep stripping it on later
same-host hops in the same chain, since net/http copies headers from the
initial request for each redirect and can otherwise restore the token.
Preserve same-host redirect behavior, retain the standard redirect limit,
and add tests for cross-host, same-host, multi-hop, and redirect-loop cases.
Accounts with shared-album zones (named CMM-*) returned these zones in
the photos changes/database listing alongside the real photo
libraries. rclone treated them as libraries and queried their albums,
but these zones have no CPLAlbumByPositionLive index so the query
failed with BAD_REQUEST / "Index has invalid data", aborting the whole
listing.
Only PrimarySync and SharedSync-* zones are photo libraries, so skip
any other zone during discovery.
See: https://forum.rclone.org/t/sync-to-icloud-gives-index-has-invalid-data-errors/53852
Non-exportable Google documents such as Google My Maps, Sites and
Forms have no export format, so rclone silently leaves them out of all
listings. During a server side move/copy this means they are quietly
left behind.
Emit a one-off notice when such a document is skipped, telling the
user the files are invisible to rclone and pointing them at
--drive-show-all-gdocs to include them in server side copies and
moves.
When moving a file rclone removed the file from its old parent by
looking the parent ID up from the path using the directory cache. When
the source contained two directories with the same name and path, the
cache could resolve to the wrong duplicate, so the removeParents
request didn't match the file's real parent.
This left the file with both its old and new parent, which fails on
shared drives with:
A shared drive item must have exactly one parent., teamDrivesParentLimit
This uses the source object's actual parent ID instead when it is
known, falling back to the path lookup only when the object has zero
or multiple parents.
At least on my macOS Sequoia 15.7.4, the system automatically adds a
com.apple.provenance xattr to files created by processes. This xattr
lacks the "user." prefix so getXattr filters it out, but the metadata
map was already allocated, resulting in an empty non-nil map being
returned instead of nil.
This caused TestMetadata/Symlink/Xattr and TestMetadata/File/Xattr
to fail because they assert the return value is nil when no user
xattrs are present.
The fix checks if the metadata map is empty after filtering and
returns nil if so.
Drime's folder rename returns success immediately but the children
listing of the renamed folder briefly returns empty before settling,
which made VFS see an empty directory right after a successful rename
and broke subsequent operations on its contents.
After a successful DirMove, poll the renamed folder's listing until it
shows the pre-rename child count, with a 30s timeout and exponential
backoff. The backend integration test already tolerated this via its
own list-consistency retries; this brings VFS into line.
Fixes#9450
Large files (sent as multipart uploads) were placed in the wrong folder
for two reasons:
- the parent folder was sent as "parent_id", but the API ignores that
and expects "parentId", so the parent was never honoured
- relativePath was sent as the full path from the drive root, which made
the server build folders from it and silently drop any "0" path
segment (e.g. ".../data/0/file" lost the "0")
Send the parent as "parentId" and use just the leaf as relativePath,
matching the working single-part upload. This also lets us remove the
now-unneeded absolute-path resolution code.
Fixes#9392
Co-authored-by: Brian King <BrianDKing@gmail.com>
The drime origin returns a malformed response (reported by Cloudflare as
a 520 error) for a literal PUT request to the file-entries update
endpoint, which broke renaming, and so server-side copy and move.
Use a POST with the X-HTTP-Method-Override: PUT header instead - the API
routes this to the same handler and it works reliably.
Also retry Cloudflare 520-524 errors which may occur transiently.
shouldRetry was a stub returning false unconditionally, which makes
protondrive the only rclone backend that disables pacer-level retries
entirely. Every other backend at minimum falls back to
fserrors.ShouldRetry(err) so genuine transport-level transients (TCP
resets, brief 5xx) get retried.
- Use errors.As to unwrap proton.APIError instead of string matching
- Retry transient storage block errors (Code=200501)
- Retry server errors (5xx, except 503)
- Skip 429 and 503 (handled by go-proton-api's resty retry layer
via catchTooManyRequests / catchRetryAfter, which honours Retry-After)
- Fall back to fserrors.ShouldRetry for non-API errors
Co-authored-by: tomholford <tomholford@users.noreply.github.com>
MEGA S4 has launched a new dedicated domain megas4.com with three new
endpoints, including their first Asia-Pacific location in Tokyo, plus
new endpoints in Paris and Barcelona. The existing Amsterdam,
Luxembourg, Montreal and Vancouver locations have been renamed onto
the new domain.
All current s4.mega.io endpoints continue to be supported and are
retained as legacy options. New accounts use the megas4.com endpoints.
Add tenant_url configuration option for accessing OneDrive/SharePoint
without admin privileges. When tenant_url is set, rclone uses the
SharePoint v2.0 API endpoint instead of the standard Microsoft Graph
API.
Co-Authored-By: KyokoMiki <161209740+kyokomiki@users.noreply.github.com>
Nick Craig-Wood