Interactively-entered passwords were run through NFKC Unicode
normalization before being obscured, which silently rewrote characters
such as ª (U+00AA) to a. The obscured password then revealed to
something different from what the user typed confusing everyone.
Normalization is only needed for the config encryption master
password, so apply it there (in SetConfigPassword) rather than in the
shared checkPassword used for backend password options.
The new "trace" dump flag attaches a net/http/httptrace ClientTrace to
each HTTP transaction and logs the connection level events - DNS
resolution, TCP connect, TLS handshake (including the negotiated TLS
version, cipher, ALPN protocol and server certificate), connection
reuse, request write and time to first response byte. Each line is
tagged with the time elapsed since the start of the transaction and the
request pointer so it can be correlated with the other dumps.
This is complementary to the existing dump flags: it shows how the
connection behaved rather than what was sent, which is useful for
debugging connectivity, DNS, TLS, proxy and keep-alive problems.
The new "errors" dump flag makes the HTTP dump conditional on the
transaction failing with a retryable error (a transport error, HTTP 429
or HTTP 5xx), so first-failure diagnostics can be captured without the
noise of dumping every transaction. The existing dump flags continue to
control what is dumped, for example --dump errors,bodies, and on its own
--dump errors dumps the headers.
TestS3Minio brings up a minio container via the fstest/testserver
framework, which exec's bash init.d scripts that shell out to docker.
This is not available on all platforms - Windows has no POSIX shell to
run the scripts, and macOS CI runners have no docker daemon - which
caused the build to fail there.
Add testy.SkipUnlessDocker to detect whether the framework can run and
skip the test when it cannot.
Previously serve s3 buffered every part of a multipart upload in memory
(in the gofakes3 S3 library) and concatenated them when the upload
completed, so memory use grew with the size of the upload.
serve s3 now streams the parts, in part-number order, into a single
PutStream upload to the underlying remote, which performs its own upload
with bounded memory. The whole file is never held in memory - memory use
is bounded by the parts in flight. This works for any remote that
supports PutStream (nearly all, including crypt) and for any part size,
so clients that don't produce uniform-sized parts (for example
PostgreSQL backup tools such as pgBarman and pgBackRest) work too.
Parts must arrive in ascending, contiguous part-number order; parts
uploaded out of order are buffered until their turn, and there is no
per-part retry (a failure aborts the whole upload). These trade-offs are
documented.
Passing --disable-multipart-streaming, or using a remote without
PutStream, reverts to buffering the parts in memory (the previous
behaviour); a one-off NOTICE is logged the first time this happens.
Fixes#7453
Run still uses a fresh local directory as the backing Fs that the
server wraps. RunWithBackend takes an extra remote name (e.g.
"TestS3Minio:") and uses a random subdirectory of that remote instead,
starting the matching fstest/testserver/init.d script on the way in
and tearing it down on the way out.
AuthProxy is only run for the local backend.
Added information about an alternative, easier way to access folders inside
"Computers" using rclone. Expanded details about folder behavior in "Computers".
Before this change, if the user changed their password or public-key
and the auth proxy script returned updated config parameters for the
backend (eg a rotated api_key) rclone would continue to re-use the old
backend with the old config parameters out of the fscache.
This was because both the VFS cache and the fs/cache key were derived
from the user name only, so a change in the user's password or
public-key did not invalidate the cached backend.
Fix this by deriving the cache key from the user plus a hash of the
password/public-key, so a credential change forces a fresh backend.
The hash uses a per-process random HMAC key so the fragment that
appears in logs cannot be brute-forced offline.
ResetCounters unconditionally restarted the average loop, spawning a
ticker goroutine that pinned the StatsInfo even when no loop had been
running before. statsGroups.delete calls ResetCounters on every removed
group, so deleting N stats groups leaked N goroutines and prevented GC
of the underlying StatsInfo objects.
Only restart the loop if it was active before the reset.
The behavior of the --track-renames and --max-delete flags combination for
bisync have confused me and some other users. So with this PR i added a
paragraph to clarify this.
Tests cover Dir size/count accounting, AttrI behaviour for files and
directories, error propagation, and Remove correctly updating ancestor
totals up the tree.
A connection string can carry global.* options which change rclone's
process-wide configuration (e.g. global.http_proxy). This is
undesirable for the rc interface which was designed to have multiple
users or connections at once. The rc interface has the `_config`
mechanism for setting request scoped global config.
This blocks global.* options on all rc paths by marking the context as
a remote control request at the rc boundaries. fs.NewFs then skips
applying global.* to the process-wide config for a marked context.
The marker is reapplied in fs.CopyConfig, which is the call rclone
uses to detach context but keep config.
global.* options still apply to the individual backend they are set
on, exactly like override.* options; they just no longer leak into the
rest of the process. Remotes created directly on the command line are
unaffected as are remotes defined in the config file.
See: GHSA-qw24-gh76-8rvv
The --rc-serve GET/HEAD file serving path accepted bracketed inline
remotes from the URL and instantiated them, so a single
unauthenticated request could run a command as the rclone user via
backend options such as webdav bearer_token_command or sftp ssh, read
arbitrary local files, or change process-wide config via global.*
options.
This was the GET/HEAD equivalent of the POST hole fixed for
CVE-2026-41179, which only guarded the rc call dispatch path.
Now, unless the rc server has authentication configured or
--rc-no-auth is set, the serve path only allows remotes already
present in the config file: inline remotes, connection string
parameters and bare local paths are rejected. Connection string
global.* options are never honoured on the serve path, even when
authenticated.
See: GHSA-qw24-gh76-8rvv
This PR fixes a bug in the WebDAV backend where directories or files could
randomly "disappear" from listings due to strict and fragile multi-status code
parsing.
Co-authored-by: bright <nako_ruru@sina.com>
Add a redirect policy to the S3 HTTP client so X-Amz-Security-Token is
removed once a redirect chain crosses hosts. Keep stripping it on later
same-host hops in the same chain, since net/http copies headers from the
initial request for each redirect and can otherwise restore the token.
Preserve same-host redirect behavior, retain the standard redirect limit,
and add tests for cross-host, same-host, multi-hop, and redirect-loop cases.
- CVE-2026-42504: mime: quadratic complexity in WordDecoder.DecodeHeader
- CVE-2026-42507: net/textproto: arbitrary input are included in errors without any escaping
- CVE-2026-27145: crypto/x509: split candidate hostname only once
Accounts with shared-album zones (named CMM-*) returned these zones in
the photos changes/database listing alongside the real photo
libraries. rclone treated them as libraries and queried their albums,
but these zones have no CPLAlbumByPositionLive index so the query
failed with BAD_REQUEST / "Index has invalid data", aborting the whole
listing.
Only PrimarySync and SharedSync-* zones are photo libraries, so skip
any other zone during discovery.
See: https://forum.rclone.org/t/sync-to-icloud-gives-index-has-invalid-data-errors/53852
The build matrix relied on setup-go's built-in cache, which keys only on
go.sum with no job differentiation. All matrix jobs computed the same
cache key and raced to save it; since cache keys are immutable, only the
first job to finish saved its cache. That winner was usually a fast job
whose build cache contained none of the cross-compiled architectures, so
the compile_all and ci_beta steps started from a cold cache on every run.
Disable setup-go's cache and add two explicit actions/cache steps to the
build matrix, the android job and the lint job:
- the module cache (~/go/pkg/mod) depends only on go.sum, so it is
shared across all jobs under a single key; it used to be duplicated in
every job's cache. The downloaded module .zip archives are pruned
before saving as they are not needed to build from the extracted
module cache, roughly halving it to ~260 MiB per OS;
- the build cache (compiled artifacts) is specific to OS, arch and Go
version, so it is kept per job, keyed on the job name.
This lets the cross-compile steps reuse per-architecture build artifacts
and keeps the total cache within the repository limit.
Measured on CI, comparing a cold-cache run against the following
warm-cache run:
other_os 23m12s -> 3m35s (compile_all 14m -> 21s)
linux 23m13s -> 12m14s (deploy 11m -> 1m37s,
race test 8m -> 4m45s)
Both jobs now finish well under 15 minutes once the cache is warm.
JSON logs reported "source":"slog/logger.go:256" instead of the real
caller. getCaller skips logging-machinery frames by file path, but
release builds use -trimpath which rewrites the standard library slog
frame's path to "log/slog/logger.go" - matching neither the "/log/" nor
the "log.go" check, so it was reported as the source. Also skip frames
whose function belongs to the log/slog package, which is immune to
-trimpath.
Non-exportable Google documents such as Google My Maps, Sites and
Forms have no export format, so rclone silently leaves them out of all
listings. During a server side move/copy this means they are quietly
left behind.
Emit a one-off notice when such a document is skipped, telling the
user the files are invisible to rclone and pointing them at
--drive-show-all-gdocs to include them in server side copies and
moves.
With cmd/mount2, reading a directory more than once returned the correct
entries on the first read but nothing on subsequent reads. Plain `ls`
triggers this: it does lseek(fd, 0, SEEK_SET) to rewind the directory
before a second getdents.
go-fuse v2.9.0 rewinds a directory stream by calling Seekdir on the
FileSeekdirer interface. dirStream did not implement it, so go-fuse
returned ENOTSUP and produced an empty listing on every read after the
first.
This implements Seekdir on dirStream: a rewind to offset 0 resets the
stream to the start, restoring correct listings on re-read. Non-zero
offsets are uncommon for in-memory listings and still return ENOTSUP,
matching go-fuse's own default. A compile-time interface assertion is
added so signature drift on future go-fuse updates is caught at build
time.
Before: second and subsequent reads of a directory returned no entries.
After: directories list correctly on every read.
See: https://github.com/hanwen/go-fuse/issues/549
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
When moving a file rclone removed the file from its old parent by
looking the parent ID up from the path using the directory cache. When
the source contained two directories with the same name and path, the
cache could resolve to the wrong duplicate, so the removeParents
request didn't match the file's real parent.
This left the file with both its old and new parent, which fails on
shared drives with:
A shared drive item must have exactly one parent., teamDrivesParentLimit
This uses the source object's actual parent ID instead when it is
known, falling back to the path lookup only when the object has zero
or multiple parents.
At least on my macOS Sequoia 15.7.4, the system automatically adds a
com.apple.provenance xattr to files created by processes. This xattr
lacks the "user." prefix so getXattr filters it out, but the metadata
map was already allocated, resulting in an empty non-nil map being
returned instead of nil.
This caused TestMetadata/Symlink/Xattr and TestMetadata/File/Xattr
to fail because they assert the return value is nil when no user
xattrs are present.
The fix checks if the metadata map is empty after filtering and
returns nil if so.
Listing the parts of a multipart upload with a MaxParts smaller than
the number of parts returned wrong PartNumber values and duplicate
parts on pages after the first, and the part number marker regressed
instead of advancing, causing clients to loop forever.
This bumps github.com/rclone/gofakes3 to v0.0.6 pick up the fix.
Drime's folder rename returns success immediately but the children
listing of the renamed folder briefly returns empty before settling,
which made VFS see an empty directory right after a successful rename
and broke subsequent operations on its contents.
After a successful DirMove, poll the renamed folder's listing until it
shows the pre-rename child count, with a 30s timeout and exponential
backoff. The backend integration test already tolerated this via its
own list-consistency retries; this brings VFS into line.
Fixes#9450
Large files (sent as multipart uploads) were placed in the wrong folder
for two reasons:
- the parent folder was sent as "parent_id", but the API ignores that
and expects "parentId", so the parent was never honoured
- relativePath was sent as the full path from the drive root, which made
the server build folders from it and silently drop any "0" path
segment (e.g. ".../data/0/file" lost the "0")
Send the parent as "parentId" and use just the leaf as relativePath,
matching the working single-part upload. This also lets us remove the
now-unneeded absolute-path resolution code.
Fixes#9392
Co-authored-by: Brian King <BrianDKing@gmail.com>
The drime origin returns a malformed response (reported by Cloudflare as
a 520 error) for a literal PUT request to the file-entries update
endpoint, which broke renaming, and so server-side copy and move.
Use a POST with the X-HTTP-Method-Override: PUT header instead - the API
routes this to the same handler and it works reliably.
Also retry Cloudflare 520-524 errors which may occur transiently.
When a SETSTAT request set the access and modification times, the
handler passed the modification time for both, discarding the
requested access time. Pass the requested access time through instead.
The VFS currently ignores the atime, but it might use it one day.
