Commit Graph

9792 Commits

Author SHA1 Message Date
IceLocke
e7b1eb774c s3: remove session token on cross-host redirects
Add a redirect policy to the S3 HTTP client so X-Amz-Security-Token is
removed once a redirect chain crosses hosts. Keep stripping it on later
same-host hops in the same chain, since net/http copies headers from the
initial request for each redirect and can otherwise restore the token.

Preserve same-host redirect behavior, retain the standard redirect limit,
and add tests for cross-host, same-host, multi-hop, and redirect-loop cases.
2026-06-03 17:58:26 +01:00
Nick Craig-Wood
c96385c280 build: fix multiple CVEs by upgrading to go1.26.4
- CVE-2026-42504: mime: quadratic complexity in WordDecoder.DecodeHeader
- CVE-2026-42507: net/textproto: arbitrary input are included in errors without any escaping
- CVE-2026-27145: crypto/x509: split candidate hostname only once
2026-06-03 12:40:28 +01:00
Nick Craig-Wood
ffd36fc2d7 iclouddrive: fix "Index has invalid data" error listing iCloud Photos
Accounts with shared-album zones (named CMM-*) returned these zones in
the photos changes/database listing alongside the real photo
libraries. rclone treated them as libraries and queried their albums,
but these zones have no CPLAlbumByPositionLive index so the query
failed with BAD_REQUEST / "Index has invalid data", aborting the whole
listing.

Only PrimarySync and SharedSync-* zones are photo libraries, so skip
any other zone during discovery.

See: https://forum.rclone.org/t/sync-to-icloud-gives-index-has-invalid-data-errors/53852
2026-06-01 17:11:23 +01:00
Nick Craig-Wood
8b981d8a69 build: cache Go build and module dirs to speed up CI
The build matrix relied on setup-go's built-in cache, which keys only on
go.sum with no job differentiation. All matrix jobs computed the same
cache key and raced to save it; since cache keys are immutable, only the
first job to finish saved its cache. That winner was usually a fast job
whose build cache contained none of the cross-compiled architectures, so
the compile_all and ci_beta steps started from a cold cache on every run.

Disable setup-go's cache and add two explicit actions/cache steps to the
build matrix, the android job and the lint job:

  - the module cache (~/go/pkg/mod) depends only on go.sum, so it is
    shared across all jobs under a single key; it used to be duplicated in
    every job's cache. The downloaded module .zip archives are pruned
    before saving as they are not needed to build from the extracted
    module cache, roughly halving it to ~260 MiB per OS;
  - the build cache (compiled artifacts) is specific to OS, arch and Go
    version, so it is kept per job, keyed on the job name.

This lets the cross-compile steps reuse per-architecture build artifacts
and keeps the total cache within the repository limit.

Measured on CI, comparing a cold-cache run against the following
warm-cache run:

    other_os    23m12s -> 3m35s    (compile_all 14m -> 21s)
    linux       23m13s -> 12m14s   (deploy 11m -> 1m37s,
                                     race test 8m -> 4m45s)

Both jobs now finish well under 15 minutes once the cache is warm.
2026-06-01 17:08:21 +01:00
Nick Craig-Wood
c34ed0a9ab log: fix wrong source file:line in JSON logs from release builds
JSON logs reported "source":"slog/logger.go:256" instead of the real
caller. getCaller skips logging-machinery frames by file path, but
release builds use -trimpath which rewrites the standard library slog
frame's path to "log/slog/logger.go" - matching neither the "/log/" nor
the "log.go" check, so it was reported as the source. Also skip frames
whose function belongs to the log/slog package, which is immune to
-trimpath.
2026-06-01 15:49:20 +01:00
Nick Craig-Wood
786e91d6a7 drive: warn when non-exportable Google documents are skipped - #9475
Non-exportable Google documents such as Google My Maps, Sites and
Forms have no export format, so rclone silently leaves them out of all
listings. During a server side move/copy this means they are quietly
left behind.

Emit a one-off notice when such a document is skipped, telling the
user the files are invisible to rclone and pointing them at
--drive-show-all-gdocs to include them in server side copies and
moves.
2026-06-01 15:41:48 +01:00
Nick Craig-Wood
e85c2e11db Add Janne Beate Bakeng to contributors 2026-06-01 15:41:48 +01:00
Nick Craig-Wood
b6b995525a Add Yakov Till to contributors 2026-06-01 15:41:48 +01:00
Janne Beate Bakeng
00bd00d83d mount2: fix empty directory listings on re-read
With cmd/mount2, reading a directory more than once returned the correct
entries on the first read but nothing on subsequent reads. Plain `ls`
triggers this: it does lseek(fd, 0, SEEK_SET) to rewind the directory
before a second getdents.

go-fuse v2.9.0 rewinds a directory stream by calling Seekdir on the
FileSeekdirer interface. dirStream did not implement it, so go-fuse
returned ENOTSUP and produced an empty listing on every read after the
first.

This implements Seekdir on dirStream: a rewind to offset 0 resets the
stream to the start, restoring correct listings on re-read. Non-zero
offsets are uncommon for in-memory listings and still return ENOTSUP,
matching go-fuse's own default. A compile-time interface assertion is
added so signature drift on future go-fuse updates is caught at build
time.

Before: second and subsequent reads of a directory returned no entries.
After: directories list correctly on every read.

See: https://github.com/hanwen/go-fuse/issues/549
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2026-06-01 12:06:03 +01:00
Yakov Till
2dbad62a11 iclouddrive: fix ADP/PCS cookie acquisition for iCloud Drive 2026-05-31 10:14:07 +01:00
Nick Craig-Wood
74436281ed drive: fix server-side move failing on shared drives with duplicate dirs - fixes #9472
When moving a file rclone removed the file from its old parent by
looking the parent ID up from the path using the directory cache. When
the source contained two directories with the same name and path, the
cache could resolve to the wrong duplicate, so the removeParents
request didn't match the file's real parent.

This left the file with both its old and new parent, which fails on
shared drives with:

    A shared drive item must have exactly one parent., teamDrivesParentLimit

This uses the source object's actual parent ID instead when it is
known, falling back to the path lookup only when the object has zero
or multiple parents.
2026-05-30 17:58:49 +01:00
Nick Craig-Wood
aec530d1e8 Add Puneet Dixit to contributors 2026-05-30 17:58:49 +01:00
Puneet Dixit
1c92cecaa3 sftp: add --sftp-encoding support 2026-05-30 11:01:20 +01:00
Leon Brocard
d2b5ff8384 local: fix getXattr returning empty map instead of nil
At least on my macOS Sequoia 15.7.4, the system automatically adds a
com.apple.provenance xattr to files created by processes. This xattr
lacks the "user." prefix so getXattr filters it out, but the metadata
map was already allocated, resulting in an empty non-nil map being
returned instead of nil.

This caused TestMetadata/Symlink/Xattr and TestMetadata/File/Xattr
to fail because they assert the return value is nil when no user
xattrs are present.

The fix checks if the metadata map is empty after filtering and
returns nil if so.
2026-05-30 10:57:55 +01:00
Nick Craig-Wood
79f42d37ff serve s3: fix multipart ListParts pagination returning wrong part numbers - fixes #9460
Listing the parts of a multipart upload with a MaxParts smaller than
the number of parts returned wrong PartNumber values and duplicate
parts on pages after the first, and the part number marker regressed
instead of advancing, causing clients to loop forever.

This bumps github.com/rclone/gofakes3 to v0.0.6 pick up the fix.
2026-05-29 17:04:01 +01:00
Nick Craig-Wood
387be8d8c0 drime: fix directory rename leaving the renamed folder empty in VFS
Drime's folder rename returns success immediately but the children
listing of the renamed folder briefly returns empty before settling,
which made VFS see an empty directory right after a successful rename
and broke subsequent operations on its contents.

After a successful DirMove, poll the renamed folder's listing until it
shows the pre-rename child count, with a 30s timeout and exponential
backoff. The backend integration test already tolerated this via its
own list-consistency retries; this brings VFS into line.

Fixes #9450
2026-05-29 12:51:26 +01:00
Nick Craig-Wood
d742334be5 Add Brian King to contributors 2026-05-29 12:51:26 +01:00
Nick Craig-Wood
c262d86ab2 Add Hakan İSMAİL to contributors 2026-05-29 12:51:26 +01:00
Hakan İSMAİL
f2832217aa rc: include OAuth authorization URL in rc config/oauthstatus response 2026-05-28 17:51:58 +01:00
Nick Craig-Wood
8a13055688 drime: remove stale mux_status field from Item
The API no longer returns mux_status (it now uses transcode_status and
stream_status). The field was unused, so just remove it.
2026-05-26 18:09:39 +01:00
Nick Craig-Wood
b6cfcb97c2 drime: remove duplicate upload_cutoff config option
The upload_cutoff option was registered twice with identical content.
Remove the second copy.
2026-05-26 18:09:28 +01:00
Nick Craig-Wood
afb29064e4 drime: fix files being uploaded to the wrong directory
Large files (sent as multipart uploads) were placed in the wrong folder
for two reasons:

- the parent folder was sent as "parent_id", but the API ignores that
  and expects "parentId", so the parent was never honoured
- relativePath was sent as the full path from the drive root, which made
  the server build folders from it and silently drop any "0" path
  segment (e.g. ".../data/0/file" lost the "0")

Send the parent as "parentId" and use just the leaf as relativePath,
matching the working single-part upload. This also lets us remove the
now-unneeded absolute-path resolution code.

Fixes #9392
Co-authored-by: Brian King <BrianDKing@gmail.com>
2026-05-26 18:09:09 +01:00
Nick Craig-Wood
8cb4bb19b0 drime: fix server-side copy and move failing with Cloudflare 520 error
The drime origin returns a malformed response (reported by Cloudflare as
a 520 error) for a literal PUT request to the file-entries update
endpoint, which broke renaming, and so server-side copy and move.

Use a POST with the X-HTTP-Method-Override: PUT header instead - the API
routes this to the same handler and it works reliably.

Also retry Cloudflare 520-524 errors which may occur transiently.
2026-05-26 18:08:42 +01:00
Nick Craig-Wood
f71bebab44 serve sftp: use the requested atime when setting file times
When a SETSTAT request set the access and modification times, the
handler passed the modification time for both, discarding the
requested access time. Pass the requested access time through instead.

The VFS currently ignores the atime, but it might use it one day.
2026-05-25 20:43:23 +01:00
Nick Craig-Wood
54cd7d6750 serve sftp: implement statvfs@openssh.com to report disk usage
The statvfs@openssh.com extension was advertised but returned an
unsupported status, so clients couldn't query the amount of free and
used space. Implement it using the VFS Statfs method, which reports the
backend's usage where the backend supports About.
2026-05-25 20:43:23 +01:00
Nick Craig-Wood
90308de5d1 serve sftp: fix truncate request being silently ignored
The SFTP serve handler ignored the size attribute of SETSTAT/FSETSTAT
requests, only acting on the modification time. This meant a client
asking to truncate a file (eg setting the final size of an upload, or
an explicit truncate) had no effect at all.

This respects the size attribute (if present) by truncating the file
to the requested size.
2026-05-25 20:43:22 +01:00
Nick Craig-Wood
4dead760dd serve sftp: fix file corruption when a client resumes an upload
The SFTP serve write handler always opened files with O_TRUNC,
ignoring the flags requested in the SFTP OPEN packet. Some clients
(notably WinSCP's "Process in Background", which resumes an upload on
a second connection) re-open the partially written file without the
truncate flag and continue writing from the offset they had reached,
relying on the existing data being preserved. Forcing O_TRUNC zeroed
that prefix, so the start of the uploaded file ended up as a block of
zero bytes.

This fix respects the requested open flags instead so a resume open
without truncate keeps the already written data intact.

See: https://forum.rclone.org/t/rclone-serve-sftp-winscp-background-mode-uploading-causes-file-corruption/53841
2026-05-25 20:43:22 +01:00
Nick Craig-Wood
6efe605b97 Add Valerij Fredriksen to contributors 2026-05-25 20:43:17 +01:00
Nick Craig-Wood
5794a22983 Add caltechustc to contributors 2026-05-25 20:43:17 +01:00
FTCHD
35b95aa2a3 rc: add config/oauthstop and config/oauthstatus to control oauth listener
rclone starts a http listener if the user is adding a remote that requires
oauth. The problem is that sometimes this breaks or the user stops midway with
no way of us knowing that. Once that happens, clients of the api break. In the
best scenario, new remotes cannot be configured without a restart.

This change adds 2 oauth related endpoints, one to get the status of the listener
and one to stop it. This allows clients to go back to normal without having to
restart rclone.

- config/oauthstop - cancels an in-progress OAuth flow and releases the port.
- config/oauthstatus - reports "running" / "stopped".
2026-05-25 19:59:09 +01:00
FTCHD
605eb30674 rc: respond with 202 if prefer-async header is passed
Make rc respond with a 202 status code (instead of 200) if `Prefer: respond-
async` was passed. Keeps backwards compatibility for current clients while also
allowing the OpenAPI schema & generators to differentiate the responses
properly.
2026-05-25 19:50:41 +01:00
Valerij Fredriksen
675806067a mount2: add --allow-idmap to advertise FUSE_ALLOW_IDMAP
Lets the kernel id-map a mount2 mount into a user namespace
(e.g. Kubernetes pods with hostUsers: false). Off by default;
requires Linux 6.12+ and implies default_permissions.
2026-05-25 17:52:36 +01:00
caltechustc
be3bfe2d19 refactor: use slices.Contains to simplify code
Signed-off-by: caltechustc <caltechustc@outlook.com>
2026-05-24 19:57:31 +01:00
Nick Craig-Wood
04d1e2563a serve nfs: allow NFS clients to mount subpaths of the served remote
Previously the Mount RPC ignored the path component of the mount
request, so `server:/sub/dir` and `server:/` both landed at the root
of the served remote. The Mount handler now cleans the requested path
with path.Clean, looks it up in the VFS and serves a billy.Filesystem
rooted at that directory, refusing the mount if the path does not
exist or is not a plain directory.

A pathRewriter cache wraps the inner handle cache so that the same
file always produces the same NFS file handle regardless of which
mount minted it (and stable across server restarts for the disk and
symlink caches). This matches the traditional NFS expectation that a
subpath mount behaves like `cd` into a subtree.

nfsmount gains a --nfs-mount-path flag (default /) so clients can
select a subpath at mount time. This replaces a latent misuse of
--volname as the NFS mount path that was previously masked by the
server ignoring it.

Fixes #9442
2026-05-24 18:09:03 +01:00
Nick Craig-Wood
761157714b Add tomholford to contributors 2026-05-24 18:08:57 +01:00
tomholford
f22a1b05a8 protondrive: implement shouldRetry instead of always returning false
shouldRetry was a stub returning false unconditionally, which makes
protondrive the only rclone backend that disables pacer-level retries
entirely. Every other backend at minimum falls back to
fserrors.ShouldRetry(err) so genuine transport-level transients (TCP
resets, brief 5xx) get retried.

- Use errors.As to unwrap proton.APIError instead of string matching
- Retry transient storage block errors (Code=200501)
- Retry server errors (5xx, except 503)
- Skip 429 and 503 (handled by go-proton-api's resty retry layer
  via catchTooManyRequests / catchRetryAfter, which honours Retry-After)
- Fall back to fserrors.ShouldRetry for non-API errors

Co-authored-by: tomholford <tomholford@users.noreply.github.com>
2026-05-24 17:50:01 +01:00
Nick Craig-Wood
7dcbc8b4f3 Changelog updates from Version v1.74.2 2026-05-23 11:34:17 +01:00
Nick Craig-Wood
05e5c05cdb Add Iizuki to contributors 2026-05-23 11:34:17 +01:00
Nick Craig-Wood
ad8a108453 lib/http: replace deprecated h2c.NewHandler with http.Server.Protocols
The golang.org/x/net/http2/h2c package was deprecated in v0.54.0 in
favour of setting the http.Server Protocols field to enable unencrypted
HTTP/2.

This replaces the h2c.NewHandler wrapping added in e863f751f with
http.Server.Protocols, which is supported by the standard library
since Go 1.24.

Note that the stdlib only supports HTTP/2 prior-knowledge on cleartext
connections, not HTTP/1.1 Upgrade: h2c negotiation. In practice clients
use prior-knowledge or require TLS, so this should not affect users.
2026-05-22 12:20:24 +01:00
Nick Craig-Wood
a03070c960 build: update golang.org/x dependencies to address multiple CVEs
Upgrade to v0.55.0 of golang.org/x/net in order to address:

- CVE-2026-42506: html: incorrect handling of namespaced elements in foreign content
- CVE-2026-39821: idna: failure to reject ASCII-only Punycode-encoded labels
- CVE-2026-42502: html: incorrect handling of HTML elements in foreign content
- CVE-2026-25680: html: denial of service when parsing arbitrary HTML
- CVE-2026-25681: html: incorrect handling of character references in DOCTYPE nodes
- CVE-2026-27136: html: duplicate attributes can cause XSS

Upgrade to version v0.52.0 of golang.org/x/crypto to address:

- CVE-2026-46598: ssh/agent: pathological inputs can lead to client panic
- CVE-2026-46597: ssh: byte arithmetic causes underflow and panic
- CVE-2026-39828: ssh: bypass of certificate restrictions
- CVE-2026-39835: ssh: server panic during CheckHostKey/Authenticate
- CVE-2026-39833: ssh/agent: key constraints not enforced
- CVE-2026-39832: ssh/agent: agent constraints dropped when forwarding keys
- CVE-2026-39827: ssh: memory leak when rejecting channels can lead to DoS
- CVE-2026-39830: ssh: client can cause server deadlock on unexpected responses
- CVE-2026-39829: ssh: pathological RSA/DSA parameters may cause DoS
- CVE-2026-39831: ssh: bypass of FIDO/U2F security keys physical interaction
- CVE-2026-39834: ssh: infinite loop on large channel writes
- CVE-2026-42508: ssh/knownhosts: auth bypass via unenforced @revoked status
- CVE-2026-46595: ssh: VerifiedPublicKeyCallback permissions skip enforcement

Upgrade to version v0.41.0 of golang.org/x/image to address:

- CVE-2026-42500: bmp: panic when reading out of bound palette index
- CVE-2026-33809: tiff: excessive resource consumption in PackBits decompression

Upgrade to version v0.45.0 of golang.org/x/sys to address:

- CVE-2026-39824: windows: integer overflow in NewNTUnicodeString
2026-05-22 11:54:29 +01:00
Iizuki
28d6b0b7b9 docs: s3: update doc link for force_path_style 2026-05-21 10:09:25 +01:00
Nick Craig-Wood
cbc64a6e6c s3: add new MEGA S4 endpoints on megas4.com including Asia-Pacific region
MEGA S4 has launched a new dedicated domain megas4.com with three new
endpoints, including their first Asia-Pacific location in Tokyo, plus
new endpoints in Paris and Barcelona. The existing Amsterdam,
Luxembourg, Montreal and Vancouver locations have been renamed onto
the new domain.

All current s4.mega.io endpoints continue to be supported and are
retained as legacy options. New accounts use the megas4.com endpoints.
2026-05-20 12:09:31 +01:00
Nick Craig-Wood
d97056cb22 docs: fix heading levels so sections appear in the table of contents
The "Import/Export of google documents" section in drive.md and the
"--b2-versions" examples section in b2.md were both at H4 instead of H3,
which excluded them from the ToC even though they are top-level topics
in their respective sections.
2026-05-20 12:09:31 +01:00
Nick Craig-Wood
3a914d9088 Add KyokoMiki to contributors 2026-05-20 12:09:31 +01:00
Nick Craig-Wood
7d4829bf7d Add TaterLi to contributors 2026-05-20 12:09:31 +01:00
Nick Craig-Wood
08bafc38ba Add William Tange to contributors 2026-05-20 12:09:31 +01:00
Nick Craig-Wood
ec011c0658 Add Lucky945H to contributors 2026-05-20 12:09:31 +01:00
Nick Craig-Wood
5c6d133389 Add Sai Asish Y to contributors 2026-05-20 12:09:31 +01:00
Nick Craig-Wood
c917ad9d36 Add Tore Anderson to contributors 2026-05-20 12:09:31 +01:00
TaterLi
7bd05530eb onedrive: add support for no admin mode
Add tenant_url configuration option for accessing OneDrive/SharePoint
without admin privileges. When tenant_url is set, rclone uses the
SharePoint v2.0 API endpoint instead of the standard Microsoft Graph
API.

Co-Authored-By: KyokoMiki <161209740+kyokomiki@users.noreply.github.com>
2026-05-19 11:21:17 +01:00