rest.URLJoin resolves its path as a URL reference according to the URL
standard, so a path beginning with `//` is a network-path reference
which replaces the host (and port and userinfo) of the base URL. That
is often the wrong behaviour.
Add rest.URLJoinRoot which joins an unescaped root onto a base URL and
refuses any root which parses with a scheme, host or userinfo of its own.
Before this change the basic, htpasswd and auth proxy authentication
middlewares let every OPTIONS request through without checking
credentials so that browser CORS preflights, which can't carry
credentials, would succeed.
However in serve webdav the OPTIONS request is answered by the WebDAV
handler whose Allow header depends on whether the path is an existing
file, an existing directory or doesn't exist, so an unauthenticated
client could discover the existence and type of any path on the served
remote even with --user/--pass or --htpasswd set.
After this change he authentication middlewares no longer special-case
OPTIONS. Instead the CORS middleware, which runs before authentication
and owns the --allow-origin setting, answers genuine CORS preflights
itself with an empty 200 carrying the CORS headers only when
--allow-origin is set.
All other OPTIONS requests need credentials like any other method, so
clients which probe with OPTIONS before authenticating will get a 401
challenge and must retry with credentials.
When pCloud redirects the browser back to rclone's local OAuth
callback server it adds a `hostname` parameter saying which regional
API host (api.pcloud.com or eapi.pcloud.com) the account lives on.
rclone used this for the token exchange and saved it in the config as
the host for all of the remote's later traffic, without checking it.
This only accepts a well formed hostname within pCloud's own domain,
lower cased, failing the config with an error otherwise, and adds a
unit test for the check. This is defence in depth for GHSA-hvpr-p4pv-4f46
The local web server which receives the OAuth callback during
`rclone config` and `rclone authorize` checks that the `state`
parameter matches the random value rclone generated when starting the
flow. The pcloud backend opted out of this check with the
`StateBlankOK` option in lib/oauthutil, which made the local server
accept a callback with no state at all.
The callback URL (http://127.0.0.1:53682/) is fixed and well known,
so while a pcloud OAuth flow was in progress any web page open in the
user's browser could complete it with a single cross-site request
carrying an attacker's authorization code. This is a CSRF
(CWE-352) on the auth callback.
This commit removes the `StateBlankOK` option so that every callback
must carry the correct state and adds a test that the local auth
server rejects callbacks with a blank, missing or wrong state.
The option was added in May 2020 (#4210) because pCloud briefly
omitted the state parameter from its redirect. The callbacks recorded
five weeks later when EU region support was added show both
api.pcloud.com and eapi.pcloud.com returning `state=` again, so the
exemption has been unnecessary since then.
See: GHSA-hvpr-p4pv-4f46
When reading a directory with --links the VFS skipped entries called "."
and ".." before removing the ".rclonelink" suffix, so an object called
"...rclonelink" (or "..rclonelink" or ".rclonelink") produced a directory
entry called ".." (or "." or "") whose path resolved to the parent
directory. The same happened for names added back from the VFS cache.
The check is now done after the suffix is removed.
See GHSA-55qp-jrwr-x827
In --links mode the local backend checked that a source name stayed inside
the destination root and only then removed the ".rclonelink" suffix from the
resulting OS path. A source entry called "...rclonelink" passed the check and
was then trimmed to "..", so the object addressed the parent of the
destination the user chose.
Writing the content of such an entry was already refused, as it is done
through an os.Root, but metadata was not: with --metadata rclone rewrote the
timestamps, ownership and extended attributes of the directory above the
destination from values supplied by the source.
The suffix is now removed from the remote name before it is joined to the
root, so the containment check applies to the name that is actually used. A
translated link must also leave a real native file name once the suffix is
removed.
A regular file whose name is exactly ".rclonelink" (or "..rclonelink" or
"...rclonelink") is not a valid translated link and is now skipped in
directory listings.
In --links mode the local backend removed the ".rclonelink" suffix from
directory names as well as file names when building the OS path. Only files
are ever translated into symlinks, so a directory called "sub.rclonelink"
addressed the sibling "sub" instead: listing it logged "Failed to read link
size", its metadata was read from the wrong place, and setting metadata on
it with --metadata failed or landed on the sibling.
Directories now use their name as is, whatever suffix it has, in the same
way DirSetModTime already does.
Every file download sends the live account authorization token and,
when configured, the SSE-C customer key headers to the download host.
With --b2-download-url that host is a CDN or proxy the user does not
operate, and the http client followed its redirects with Go's default
policy:
- Go only strips Authorization when the redirect target is not the same
host or a subdomain, and never looks at the scheme, so a same-host
HTTPS to HTTP redirect replayed the full-capability B2 token in
cleartext.
- Go has no idea that the X-Bz-Server-Side-Encryption-Customer-* headers
carry the customer encryption key, so it forwarded them to any
redirect target including a different host.
The B2 client now uses rest.StripHeadersOnCrossHostRedirectFn, so it
refuses to follow a redirect which downgrades HTTPS to HTTP and strips
the Authorization and SSE-C headers from any redirect which leaves the
host and port of the original request, matching the protection already
given to the S3, WebDAV and HTTP backends.
Fixes GHSA-gpw7-77cq-9f72
The s3 and http backends each carried their own copy of the same
CheckRedirect policy. This moves it into lib/rest as
StripHeadersOnCrossHostRedirectFn, so that any backend which sends
credentials in headers to a host it does not control can use the same
policy. It logs at debug level when it strips headers so that a
resulting 401 from the redirect target can be diagnosed. A header
starting with "*" is matched without canonicalising, as that is how
SetHeader and Opts.ExtraHeaders send it.
This changes the s3 policy in two edge cases:
- It compared the raw URL host string, so a redirect to the same host
in different case or with an explicit default port stripped the
credentials. It now uses SameHost like the http backend, which treats
those as the same host.
- The downgrade check compared each hop with the previous one. It now
compares with the original request like the other backends, so a
chain which starts on plaintext HTTP may return to it.
The behaviour tests move to lib/rest. The backend tests which check the
policy is wired into each client are kept.
- CVE-2026-97032: net/http: HTTP/2 server crash due to HPACK encoder race
- CVE-2026-78659: net/http: HTTP/2 server memory exhaustion due to Trailer headers
- CVE-2026-97031: crypto/tls: reject malformed ECH outer extension references
- CVE-2026-94444: cmd/go: checksum bypass for golang.org/fips140
- CVE-2026-94447: cmd/go: checksum database bypass for golang.org/toolchain
- CVE-2026-94448: html/template: reset context tracking on consecutive template expressions
- CVE-2026-97030: html/template: recognize |yield| as regexp preceder keyword
- CVE-2026-94440: net/textproto, mime/multipart: memory limit bypass when parsing MIME headers
- CVE-2026-56866: net/http: HTTP/1 client connection desynchronization after CONNECT rejection
- CVE-2026-94439: net/http: HTTP/1 server connection desynchronization after 2xx CONNECT response
- CVE-2026-78669: net/http: excessive CPU consumption from repeated initial window changes
- CVE-2026-78660: net/http: HTTP/2 transport accepts malformed framing-related headers
- CVE-2026-56857: os: Root.Mkdir(All) can follow junctions out of the root on Windows
- CVE-2026-78667: net/http: lack of limit on size of parsed Range headers
- CVE-2026-78663: net/http: double flow control refund on HTTP/2 server streams
This also updates the go1.26 test job to go1.26.9 which contains the
same fixes.
Nextcloud chunk uploads currently return a retryable error when a
chunk PUT fails. The outer low-level retry then restarts the whole
upload, removing the temporary upload directory and sending previously
accepted chunks again.
Retry each chunk in place using the configured low-level retry budget,
rewinding its repeatable body before each attempt. Once that budget is
exhausted, prevent the outer retry from restarting the entire object.
WriteChunk checksums the chunk with io.Copy before sending it. For a
reader that implements neither WriterTo nor a memory backed Read - a
local file opened by a multi-thread copy - io.Copy falls back to its own
32 KiB buffer, so the source is read in 32 KiB pieces and the disk sees
thousands of requests per second where it should see hundreds.
Hand io.CopyBuffer a 1 MiB buffer instead. It is allocated for the
checksum pass rather than taken from the pool, where it could wait for
ever on a --max-buffer-memory below the page size. Readers that do
implement WriterTo, such as pool.RW, keep their fast path and don't get
a buffer.
FsOpenChunkWriter wrote three chunks of its own fixed sizes whatever
ChunkWriterInfo.ChunkSize the backend returned. That happened to be
the size the s3 backend uses by default, so the test passed there, but
a chunk writer which works out where a chunk goes from its number needs
every chunk but the last to be exactly the size it asked for - as a
multi-thread copy writes them - and the test could neither exercise nor
catch that.
Cut the file into chunks of info.ChunkSize, the last one short, still
written out of order. Where the backend lets the test set its chunk
size, ask for 5 MiB first so the file takes several chunks. Where it
doesn't and the chunks leave the file in fewer than three, start again
with a file of two full chunks and a short one, for chunks up to
128 MiB.
A failed delta poll used to wipe the stored resume token: changeNotifyRunner
returned a named nextDeltaToken which stayed empty on error, and the caller
stored it unconditionally. Later polls then asked for /root/delta?token=
instead of resuming, so notifications stopped until the mount was restarted.
changeNotifyNextChange also called the API without the pacer, so a single
transient 503 was enough to trigger this.
While fixing that, walk @odata.nextLink as well. A change set spanning more
than one page returns nextLink without a deltaLink, and the token parsed from
the missing deltaLink was empty - changes were dropped even when no error
occurred.
Fixes#10009
Before this change the parent process kept the remote control and
metrics servers it started in initConfig listening, so the daemon child
could not bind the same addresses and exited.
After this change the parent shuts those servers down just before
daemonizing, and the child starts them as usual.
A symlink in an archive was listed as an ordinary file holding the link
target by the zip backend, and dropped entirely by the squashfs one.
Symlinks are now skipped unless -l/--links is in use, as they are on the
other backends. With that flag they are listed with the .rclonelink
suffix and read back as the path they point at, so they can be copied to
a destination which supports symlinks.
This changes what the zip backend lists by default: a symlink used to
appear as a small file whose contents were the link target, and is now
skipped with a NOTICE naming the flag.
Fixes#9569
If a backend's ListR returned a directory after objects inside it,
the directory was listed twice when using --max-depth or filters. This
happens with azureblob on storage accounts with a hierarchical
namespace.
When a backend saved config, for example a refreshed OAuth token, for
a remote which wasn't in the config file, a section with just that
value was created in the config file for environment variable remotes
and the config file was rewritten for on the fly remotes.
Config is now only saved if the remote is in the config file.
Listing "dir/" on a storage account with a hierarchical namespace
returns the file "dir" if it exists, so a file was listed as a
directory containing itself.
On storage accounts with a hierarchical namespace, directories can be
read as blobs so NewObject on a directory returned fs.ErrorNotAFile
rather than fs.ErrorObjectNotFound.
Storage accounts with a hierarchical namespace (ADLS Gen2) have real
directories. Rclone could not remove these - with directory_markers it
failed with "InvalidUri" and without it rmdir silently did nothing.
Directories are also left behind when the last file in them is
deleted.
This adds the hns option which is read from the storage account if
not set and saved in the config file. If set, rclone:
- supports empty directories whatever the directory_markers setting
- creates directories with Mkdir
- removes directories with Rmdir
- doesn't create directory markers for parent directories on upload
With --auth-proxy, when several logins with the same credentials
arrived together and none was in the cache yet, as when an FTP client
opens several connections at once, each one ran the auth proxy program
and took its own reference to the backend. Only one of them was
remembered, so the others were never given back and the backend was
never shut down.
Logins which arrive together now share one run of the auth proxy
program and one reference to the backend.
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. With --vfs-cache-mode writes or full, files
are uploaded to the backend after the transfer which wrote them has
finished, which didn't count as a use. So an upload which hadn't
finished 5 minutes after the user's last command (or disconnection for
serve sftp) was stopped, leaving the file in the VFS cache but not on
the backend.
The backend is now kept until it has no files open for write and
nothing in the VFS cache waiting to be uploaded.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
Busy returns true if the VFS has files open for write or files in the
VFS cache which are open or waiting to be uploaded. It is for code
which wants to shut a VFS down only once that wouldn't lose any work.
An entry is expired when it hasn't been used for a while, but the user
of the cache may know it is still doing something in the background.
SetCanExpire sets a function which is asked before expiring an entry
and can keep it in the cache until it is ready to go.
The function is called without the cache locked as it may take a
while, so an entry which is used in the meantime isn't expired.
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each request counted as a
use, so an upload or download lasting longer than that had its backend
shut down under it and failed.
Each request now holds the backend it uses until it has been served.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each request counted as a
use, so a download lasting longer than that had its backend shut down
under it and failed.
Each request now holds the backend it uses until it has been served.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
With --auth-proxy the backend of each user is shut down once it has
been unused for 5 minutes. Only the start of each FTP command counted
as a use, so an upload or download lasting longer than that had its
backend shut down under it and failed with "context canceled".
Each FTP command now holds the backend it uses until it has finished,
which for a download is when the file is closed.
This was introduced in v1.75.1 by
f425f8d46 serve: refactor VFS and proxy handling into Provider
Add a new backend for dosya.dev, a cloud file storage, sharing, and
synchronization platform with workspace-based multi-tenancy.
Features:
- Server-side copy, move, rename (files and directories)
- Purge (recursive directory delete)
- Recursive listing (ListR) for --fast-list
- Multipart upload for large files (>10MB)
- Public link sharing via rclone link
- About (workspace storage quota)
- MIME type detection
- Empty directory support
- Workspace-based multi-tenancy (one remote per workspace)
- Download via presigned R2 URLs with Range header support
Co-authored-by: firatkaya <firat@netiket.com.tr>
When an upload failed during the transfer, the partial file was
removed but its handle was never closed. The server kept the deleted
file open, so its space stayed allocated until the pooled connection
was closed, which in rcd, mount or serve could be until rclone exited.
This closes the handle before removing the partial file.
Tests in fs/operations, fs/sync, cmd/bisync, cmd/gitannex and vfs are
run by the integration tests against every backend, so they need to
cope with backends with limited functionality, ideally by skipping on
the Fs.Features() flags.
Files.com now treats the replacement characters rclone uses for / and
\ in file names (／ and ＼) as if they were the originals, and the
names ． and ．． as if they were . and .., and rejects them all with
"Invalid path". There is no other way of encoding these names, so
document the restriction and ignore the integration tests for them.
When an upload was stopped part way through by --max-transfer, HiDrive
kept the part of the file it had received, leaving a truncated file at
the destination, or replacing an existing file with a truncated one.
This was introduced in e8f421d28 which accounted the buffered start of
each upload as it was sent rather than as it was read, so the transfer
limit could abort the request which creates the file half way through.
Account the requests which create or replace a file in one go as they
are buffered once more, so the limit is hit before anything is sent.
The chunks of larger uploads are still accounted as they are sent.
ownCloud 10.16 answers a PROPFIND for a path which doesn't exist with
a 207 Multi-Status response, rather than a 404, with a body which
starts a multistatus document then appends a Sabre NotFound error to
it. This isn't valid XML so rclone failed with
read metadata failed: XML syntax error on line 2: expected attribute name in element
whenever it was pointed at a directory which didn't exist yet.
Detect the NotFound error in the response and treat it as a 404.
Koofr refuses to download the HTML listings written by the index tests
with "FileBlocked: File download restricted due to possible dangerous
content" and OpenDrive refuses with 403 Forbidden, so the tests can't
read back what they wrote. The index tests which write other formats
still run.
With meta_format = none, listing a directory which contained the
temporary chunks of an interrupted upload, but no completed chunks for
that file, panicked with "invalid chunked object".
The listing made a placeholder object for the file on seeing a
temporary chunk, which then had no chunks in it when it was validated.
Only make the placeholder on seeing a data chunk, so files which have
nothing but temporary chunks are ignored as they are when metadata is
in use.
- Only check the MIME type of the listings on backends which declare
both ReadMimeType and WriteMimeType, and ignore its parameters, as
many backends report a MIME type they chose themselves or drop the
charset.
- Write the test file before turning --dry-run on. Chunker finishes
an upload with operations.Move, which honours --dry-run, so it left
temporary chunks behind which stopped the test directory being
removed and made the tests following fail.
- Don't check the transfer and delete counts on chunker, as moving
its chunks into place is counted in the same stats.
- Don't expect a new file to change the time of the directories above
it on backends which can't set modification times, as it may get the
same time as the files already there.
The Blob Listing with Apache Arrow feature is now public and shipped
in azblob v1.8.1, so the temporary backend/azureblob/arrowlist package
which implemented it on top of the previous SDK has been removed and
the backend now uses the SDK's own listing pager with ResponseFormat
set to Arrow.
As the SDK client handles every credential type this also makes Arrow
and parallel listing work with connection_string auth, and the
use_arrow_list and list_parallelism options are no longer hidden.
Nick Craig-Wood