The s3 and http backends each carried their own copy of the same
CheckRedirect policy. This moves it into lib/rest as
StripHeadersOnCrossHostRedirectFn, so that any backend which sends
credentials in headers to a host it does not control can use the same
policy. It logs at debug level when it strips headers so that a
resulting 401 from the redirect target can be diagnosed. A header
starting with "*" is matched without canonicalising, as that is how
SetHeader and Opts.ExtraHeaders send it.
This changes the s3 policy in two edge cases:
- It compared the raw URL host string, so a redirect to the same host
in different case or with an explicit default port stripped the
credentials. It now uses SameHost like the http backend, which treats
those as the same host.
- The downgrade check compared each hop with the previous one. It now
compares with the original request like the other backends, so a
chain which starts on plaintext HTTP may return to it.
The behaviour tests move to lib/rest. The backend tests which check the
policy is wired into each client are kept.
(cherry picked from commit cae10713e8)
- CVE-2026-97032: net/http: HTTP/2 server crash due to HPACK encoder race
- CVE-2026-78659: net/http: HTTP/2 server memory exhaustion due to Trailer headers
- CVE-2026-97031: crypto/tls: reject malformed ECH outer extension references
- CVE-2026-94444: cmd/go: checksum bypass for golang.org/fips140
- CVE-2026-94447: cmd/go: checksum database bypass for golang.org/toolchain
- CVE-2026-94448: html/template: reset context tracking on consecutive template expressions
- CVE-2026-97030: html/template: recognize |yield| as regexp preceder keyword
- CVE-2026-94440: net/textproto, mime/multipart: memory limit bypass when parsing MIME headers
- CVE-2026-56866: net/http: HTTP/1 client connection desynchronization after CONNECT rejection
- CVE-2026-94439: net/http: HTTP/1 server connection desynchronization after 2xx CONNECT response
- CVE-2026-78669: net/http: excessive CPU consumption from repeated initial window changes
- CVE-2026-78660: net/http: HTTP/2 transport accepts malformed framing-related headers
- CVE-2026-56857: os: Root.Mkdir(All) can follow junctions out of the root on Windows
- CVE-2026-78667: net/http: lack of limit on size of parsed Range headers
- CVE-2026-78663: net/http: double flow control refund on HTTP/2 server streams
This also updates the go1.26 test job to go1.26.9.
TestCopyFileCopyDestImmutable relies on --immutable being enforced for
copyto, moveto and single file copies, which v1.75 doesn't do, so it
failed wherever the remote supports server-side copy, such as the
local backend on macOS.
The --copy-dest --immutable fix it was picked with still applies to
sync and copy and is tested by TestSyncCopyDestImmutable.
Before this change the parent process kept the remote control and
metrics servers it started in initConfig listening, so the daemon child
could not bind the same addresses and exited.
After this change the parent shuts those servers down just before
daemonizing, and the child starts them as usual.
(cherry picked from commit 4532a88838)
With --auth-proxy, when several logins with the same credentials
arrived together and none was in the cache yet, as when an FTP client
opens several connections at once, each one ran the auth proxy program
and took its own reference to the backend. Only one of them was
remembered, so the others were never given back and the backend was
never shut down.
Logins which arrive together now share one run of the auth proxy
program and one reference to the backend.
(cherry picked from commit 2e9c06c84d)
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. With --vfs-cache-mode writes or full, files
are uploaded to the backend after the transfer which wrote them has
finished, which didn't count as a use. So an upload which hadn't
finished 5 minutes after the user's last command (or disconnection for
serve sftp) was stopped, leaving the file in the VFS cache but not on
the backend.
The backend is now kept until it has no files open for write and
nothing in the VFS cache waiting to be uploaded.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
(cherry picked from commit 0caf24af50)
Busy returns true if the VFS has files open for write or files in the
VFS cache which are open or waiting to be uploaded. It is for code
which wants to shut a VFS down only once that wouldn't lose any work.
(cherry picked from commit 4fa2a5b02c)
An entry is expired when it hasn't been used for a while, but the user
of the cache may know it is still doing something in the background.
SetCanExpire sets a function which is asked before expiring an entry
and can keep it in the cache until it is ready to go.
The function is called without the cache locked as it may take a
while, so an entry which is used in the meantime isn't expired.
(cherry picked from commit 49d8566ec5)
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each request counted as a
use, so an upload or download lasting longer than that had its backend
shut down under it and failed.
Each request now holds the backend it uses until it has been served.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
(cherry picked from commit 7e6d6fa838)
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each request counted as a
use, so a download lasting longer than that had its backend shut down
under it and failed.
Each request now holds the backend it uses until it has been served.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
(cherry picked from commit 8457da3c8a)
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each FTP command counted
as a use, so an upload or download lasting longer than that had its
backend shut down under it and failed with "context canceled".
Each FTP command now holds the backend it uses until it has finished,
which for a download is when the file is closed.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
(cherry picked from commit ee0a70f875)
When an upload failed during the transfer, the partial file was
removed but its handle was never closed. The server kept the deleted
file open, so its space stayed allocated until the pooled connection
was closed, which in rcd, mount or serve could be until rclone exited.
This closes the handle before removing the partial file.
(cherry picked from commit d545615c16)
Files.com now treats the replacement characters rclone uses for / and
\ in file names (／ and ＼) as if they were the originals, and the
names ． and ．． as if they were . and .., and rejects them all with
"Invalid path". There is no other way of encoding these names, so
document the restriction and ignore the integration tests for them.
(cherry picked from commit ed92e4446e)
ownCloud 10.16 answers a PROPFIND for a path which doesn't exist with
a 207 Multi-Status response, rather than a 404, with a body which
starts a multistatus document then appends a Sabre NotFound error to
it. This isn't valid XML so rclone failed with
read metadata failed: XML syntax error on line 2: expected attribute name in element
whenever it was pointed at a directory which didn't exist yet.
Detect the NotFound error in the response and treat it as a 404.
(cherry picked from commit f217c8a2aa)
With meta_format = none, listing a directory which contained the
temporary chunks of an interrupted upload, but no completed chunks for
that file, panicked with "invalid chunked object".
The listing made a placeholder object for the file on seeing a
temporary chunk, which then had no chunks in it when it was validated.
Only make the placeholder on seeing a data chunk, so files which have
nothing but temporary chunks are ignored as they are when metadata is
in use.
(cherry picked from commit 475997b6de)
The Blob Listing with Apache Arrow feature is now public and shipped
in azblob v1.8.1, so the temporary backend/azureblob/arrowlist package
which implemented it on top of the previous SDK has been removed and
the backend now uses the SDK's own listing pager with ResponseFormat
set to Arrow.
As the SDK client handles every credential type this also makes Arrow
and parallel listing work with connection_string auth, and the
use_arrow_list and list_parallelism options are no longer hidden.
(cherry picked from commit 3b9f0a05b5)
Pointing the archive backend at a file inside a squashfs image, for
example `rclone cat :archive:image.sqfs/dir/file.txt`, listed every
file in the directory containing it instead of just that file. The zip
archiver already behaved correctly.
The squashfs archiver now remembers the file the root points at and
only exposes that one, as zip does. Its Root() includes the file so
that the fs cache does not hand back the Fs for the whole directory in
its place.
(cherry picked from commit 67afe55a83)
Setting the modtime of a closed file refreshed the cache item's
fingerprint before the new modtime had been applied to the remote
object. The next open then saw a fingerprint mismatch, logged "removed
cache file as stale (remote is different)" and downloaded the whole
file again, even though only the modtime had changed.
Refresh the cache fingerprint again once the remote modtime has been
set.
(cherry picked from commit 9380daed0d)
When a file was reopened within the --vfs-handle-caching grace period
(default 5s) after its remote fingerprint changed - for example after a
touch or any other modtime change - _checkObject removed the stale
cache file and open recreated an empty one, but nothing sized it. The
next GetSize stat'd the empty file and set the item size to 0, so
ReadAt's _ensure clamped the request to nothing and skipped the
download, then the size check zero-extended the file and returned the
zeros. Reads through a mount returned the correct length but zeros for
the first read (128 KiB through FUSE).
This is easy to hit with git on a --vfs-cache-mode full mount: git
freshens pack files with utime, and a concurrent reader then sees
"not a GIT packfile".
Size the recreated cache file from the object before opening it, as a
normal open does.
(cherry picked from commit e0fb01084d)
Shutdown was calling expiryTimer.Stop without holding ts.mu, so it could
panic if OnExpiry had not created the timer yet, and raced the write in
OnExpiry. Hold the lock and skip Stop when the timer is still nil.
Fixes#9980
(cherry picked from commit 80e462d48c)
Until the first time slot of the week, the timetable used its last entry
as the limit carried over from the week before. That is only the latest
slot of the week when the entries are in order, so a timetable written
weekend first, like "Sat-00:00,off Mon-00:00,1M", limited Sunday to 1M
instead of leaving it unlimited.
Carry over the latest time slot of the week instead.
(cherry picked from commit 124f3bd830)
When listing with --s3-versions or --s3-version-at the storage class
of each object was dropped, so it read as STANDARD unless the object's
metadata was fetched separately. This meant backend restore skipped
objects in GLACIER or DEEP_ARCHIVE with "Not GLACIER or DEEP_ARCHIVE
or INTELLIGENT_TIERING storage class", and lsf --format T showed the
wrong tier.
This happened because ObjectVersion.StorageClass has a different type
from Object.StorageClass so the generated setFrom helper does not copy
it. Convert it explicitly after the setFrom call.
(cherry picked from commit e5eaf414f0)
The Docker image had no /etc/mime.types, so MIME types came only from
Go's built-in table plus rclone's small extra list, and many
extensions uploaded as application/octet-stream.
This installs Alpine's mailcap package to fix the problem.
(cherry picked from commit 3cdf855547)
To find the latest patch release of a minor version, for example
`rclone selfupdate --version 1.75`, selfupdate searched the listing of
downloads.rclone.org for href="./vX.Y.Z/". The leading ./ is a detail
of how Caddy's file server writes its links. A listing which linked to
the same directories as "vX.Y.Z/", which is an equally valid relative
URL, made selfupdate fail with "could not find the minor release".
This makes the ./ optional in the pattern, so selfupdate no longer
depends on which program wrote the listing, and takes the version from
a capture group rather than from fixed offsets into the match.
The listings written by rclone index now include the ./ for the
benefit of rclone versions without this fix, so this is to remove the
dependency for the future.
(cherry picked from commit 8cd43746a3)
The temporary directory contains a drive letter colon so it needs
quoting in the crypt connection string, otherwise the password
parameter is dropped.
(cherry picked from commit 9dc8b71ae9)
The newer revive flags an exported function returning an unexported
type and a redundant type in a var declaration.
(cherry picked from commit 881cedd348)
MinIO no longer publishes images on quay.io (nor Docker Hub or ghcr.io)
so pulling quay.io/minio/minio returns "unauthorized". pgsty/minio is a
maintained community fork with the same entrypoint layout.
(cherry picked from commit 220c65f81d)
ENETUNREACH and ENETDOWN were not in the list of retriable errors on
non-Windows platforms, so a single failed connection aborted the
transfer instead of being retried as a low level retry. The Windows
list already includes the equivalent WSAENETUNREACH and WSAENETDOWN.
This is easy to hit on a host with IPv6 enabled but no IPv6 route: if
the A lookup fails transiently while the AAAA lookup succeeds, the only
address to dial is IPv6 and the connect fails with ENETUNREACH. A retry
resolves again and normally succeeds.
(cherry picked from commit 94e319d2fa)
BwPair.String printed a sub-KiB bandwidth as a bare number, and Set
reads a bare number as KiB, so a rate under 1 KiB grew by a factor of
1024 whenever it went through a string. rc core/bwlimit reports the
current rate in that format and accepts it back, so reading the limit
and setting it again raised it.
63c4fef27 fixed the same corruption for config values by suffixing bare
numbers with B inside Option.String. Move that into a SizeSuffix method
and use it from BwPair.String as well.
(cherry picked from commit 60bdc6d454)
Use mime.FormatMediaType instead of unescaped string concatenation so a
directory name containing a double quote cannot inject extra disposition
parameters.
Fixes#9962
(cherry picked from commit 14a1359c96)
Use mime.FormatMediaType instead of unescaped string concatenation so a
directory name containing a double quote cannot inject extra disposition
parameters.
Fixes#9962
(cherry picked from commit 661484f36c)
Copying an object onto itself, as clients do to replace its metadata,
reported success when the object didn't exist, and copying a missing
object to a different key failed with an internal error. Both now fail
with NoSuchKey as S3 does.
(cherry picked from commit ce5351c52e)
With --etag-hash auto the hash for the ETags was chosen once from the
remote serve s3 was started with. With --auth-proxy there is no such
remote so serve s3 crashed on startup, and when started by the rc it
was the wrong remote, so users whose backends lacked that hash got no
ETags.
The hash is now chosen from the backend of the user making each
request.
(cherry picked from commit 746eac73ef)
The metadata of every object uploaded was kept in memory for as long as
the server ran, even after the object was deleted, so it grew without
limit and the metadata of a deleted object reappeared on any object
later created at the same key other than through serve s3.
Deleting an object now forgets its metadata.
(cherry picked from commit c350f8ebbf)
Listing a prefix with no delimiter walked the entire subtree below it into
memory and only then sliced out the requested page, so every page of a
listing cost a full traversal of the tree.
Walk the tree lazily instead, stopping as soon as the page is full and
skipping the subtrees an earlier page already returned. A page now costs a
number of directory reads proportional to the keys it returns rather than to
the size of the subtree, and ETags are computed only for the objects that
are actually returned.
Entries are emitted in the order their keys have in a flat keyspace, with a
directory sorting as if it carried its trailing slash, so that "a.txt" comes
before "a/b" as it does in a real S3 bucket. Without this a resumed listing
would silently skip keys across a page boundary.
(cherry picked from commit 04697cc02a)
When the last user of a VFS shut it down at the same time as a new
user of the same remote asked for one, the new user could be given the
VFS being shut down, with its cache and background tasks stopped.
Only a VFS which is still in use is now reused, otherwise a new one is
made.
(cherry picked from commit c62aa2adc9)
A VFS is shut down when the last of the references to it from New is
given back with Shutdown. Hold takes another reference, for code using
a VFS it didn't create itself, which may otherwise be shut down under
it.
(cherry picked from commit 1dab0f3abd)
When a backend rejected the multipart upload as too small, the subtest
skipped without removing its local source file, so the following
upload subtests failed their local listing checks.
(cherry picked from commit ee9c228012)
Moving a directory can take longer than the API gateway allows, so the
request fails with a 520 or 502 error even though the move completes.
The retry then failed with "Folder ... was already moved to that
location (status 409)".
Treat that error as success.
(cherry picked from commit 0c52b183d0)
The Internxt API serves listings from read replicas which lag behind
writes, so for a short while after files or directories are moved or
deleted they can still be listed in their old location.
This caused removing a directory which had just been emptied to fail
with "directory not empty" (eg when moving a directory without server
side directory moves or purging a directory) and a directory which had
just been moved to be found in its old location.
Remember the directories this process has moved or deleted and ignore
directory and file entries which contradict that when listing, finding
directories and checking a directory is empty before removing it.
(cherry picked from commit 7ff5da8517)
Moving a file into a directory which had just been created failed with
"Not Found (status 404)", and moving a file over one which had just been
deleted (as sync does with --backup-dir and --suffix) failed with "A file
with the same name already exists in destination folder (status 409)".
The API checks moves against read replicas which lag behind writes, so
retry these errors until the replicas catch up.
(cherry picked from commit ee26daecd8)
The Internxt API serves lookups and listings from read replicas which
lag behind writes, so for a short while after a file is moved it can
still be returned from its old location.
When sync moved a file into the backup location and then uploaded its
replacement, the upload could find the moved file under its old name
and overwrite it. Overwriting renames the existing file by UUID and
deletes it once the upload succeeds, so the file that had just been
moved into the backup location was deleted.
Remember the files this process has moved or deleted for a minute and
ignore lookups and listing entries which contradict that.
(cherry picked from commit f068abd607)
With hard_delete set, deleted files and directories carried on being
listed for about a second afterwards because the Drime server doesn't
invalidate its cache of the parent folder listing when entries are
deleted forever. This made removing a directory straight after
emptying it fail with "directory not empty".
Moving an entry to the trash does invalidate the cache, so with
hard_delete set rclone now moves the entry to the trash first and then
deletes it forever.
(cherry picked from commit a2e2b73725)
When server-side copying to a destination which already existed, the
Drime server gave the copy the name "name (1)" and rclone only renamed
it if the source and destination leaf names differed. The existing file
was then deleted leaving the copy under the wrong name.
The server refuses to rename an entry to a name which is already in
use, so this removes the existing file straight after the copy, then
renames the copy whenever its name differs from the destination name.
(cherry picked from commit eb10c48a17)
The premiumize.me upload server has started truncating the multipart
file name at the first ";", so uploading "a;b.txt" created a file
called "a" and the upload then failed with "object not found".
Directory creation and renames still accept ";", so files whose names
contain ";" are now uploaded under a temporary name and renamed into
place. The encoding is left unchanged so existing files and
directories containing ";" remain accessible.
(cherry picked from commit 7c18e1eb86)
Nick Craig-Wood