Commit Graph

9736 Commits

Author SHA1 Message Date
Nick Craig-Wood
dff3a0731e build: cache Go build and module dirs to speed up CI
The build matrix relied on setup-go's built-in cache, which keys only on
go.sum with no job differentiation. All matrix jobs computed the same
cache key and raced to save it; since cache keys are immutable, only the
first job to finish saved its cache. That winner was usually a fast job
whose build cache contained none of the cross-compiled architectures, so
the compile_all and ci_beta steps started from a cold cache on every run.

Disable setup-go's cache and add two explicit actions/cache steps to the
build matrix, the android job and the lint job:

  - the module cache (~/go/pkg/mod) depends only on go.sum, so it is
    shared across all jobs under a single key; it used to be duplicated in
    every job's cache. The downloaded module .zip archives are pruned
    before saving as they are not needed to build from the extracted
    module cache, roughly halving it to ~260 MiB per OS;
  - the build cache (compiled artifacts) is specific to OS, arch and Go
    version, so it is kept per job, keyed on the job name.

This lets the cross-compile steps reuse per-architecture build artifacts
and keeps the total cache within the repository limit.

Measured on CI, comparing a cold-cache run against the following
warm-cache run:

    other_os    23m12s -> 3m35s    (compile_all 14m -> 21s)
    linux       23m13s -> 12m14s   (deploy 11m -> 1m37s,
                                     race test 8m -> 4m45s)

Both jobs now finish well under 15 minutes once the cache is warm.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
5f5e2ef9cf log: fix wrong source file:line in JSON logs from release builds
JSON logs reported "source":"slog/logger.go:256" instead of the real
caller. getCaller skips logging-machinery frames by file path, but
release builds use -trimpath which rewrites the standard library slog
frame's path to "log/slog/logger.go" - matching neither the "/log/" nor
the "log.go" check, so it was reported as the source. Also skip frames
whose function belongs to the log/slog package, which is immune to
-trimpath.
2026-06-03 17:51:47 +01:00
Janne Beate Bakeng
d4397faa94 mount2: fix empty directory listings on re-read
With cmd/mount2, reading a directory more than once returned the correct
entries on the first read but nothing on subsequent reads. Plain `ls`
triggers this: it does lseek(fd, 0, SEEK_SET) to rewind the directory
before a second getdents.

go-fuse v2.9.0 rewinds a directory stream by calling Seekdir on the
FileSeekdirer interface. dirStream did not implement it, so go-fuse
returned ENOTSUP and produced an empty listing on every read after the
first.

This implements Seekdir on dirStream: a rewind to offset 0 resets the
stream to the start, restoring correct listings on re-read. Non-zero
offsets are uncommon for in-memory listings and still return ENOTSUP,
matching go-fuse's own default. A compile-time interface assertion is
added so signature drift on future go-fuse updates is caught at build
time.

Before: second and subsequent reads of a directory returned no entries.
After: directories list correctly on every read.

See: https://github.com/hanwen/go-fuse/issues/549
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2026-06-03 17:51:47 +01:00
Yakov Till
7324ee850a iclouddrive: fix ADP/PCS cookie acquisition for iCloud Drive 2026-06-03 17:51:47 +01:00
Nick Craig-Wood
3c54a37488 drive: fix server-side move failing on shared drives with duplicate dirs - fixes #9472
When moving a file rclone removed the file from its old parent by
looking the parent ID up from the path using the directory cache. When
the source contained two directories with the same name and path, the
cache could resolve to the wrong duplicate, so the removeParents
request didn't match the file's real parent.

This left the file with both its old and new parent, which fails on
shared drives with:

    A shared drive item must have exactly one parent., teamDrivesParentLimit

This uses the source object's actual parent ID instead when it is
known, falling back to the path lookup only when the object has zero
or multiple parents.
2026-06-03 17:51:47 +01:00
Leon Brocard
a2f8b493ed local: fix getXattr returning empty map instead of nil
At least on my macOS Sequoia 15.7.4, the system automatically adds a
com.apple.provenance xattr to files created by processes. This xattr
lacks the "user." prefix so getXattr filters it out, but the metadata
map was already allocated, resulting in an empty non-nil map being
returned instead of nil.

This caused TestMetadata/Symlink/Xattr and TestMetadata/File/Xattr
to fail because they assert the return value is nil when no user
xattrs are present.

The fix checks if the metadata map is empty after filtering and
returns nil if so.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
641702fc7f serve s3: fix multipart ListParts pagination returning wrong part numbers - fixes #9460
Listing the parts of a multipart upload with a MaxParts smaller than
the number of parts returned wrong PartNumber values and duplicate
parts on pages after the first, and the part number marker regressed
instead of advancing, causing clients to loop forever.

This bumps github.com/rclone/gofakes3 to v0.0.6 pick up the fix.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
f25d9e2bf3 drime: fix directory rename leaving the renamed folder empty in VFS
Drime's folder rename returns success immediately but the children
listing of the renamed folder briefly returns empty before settling,
which made VFS see an empty directory right after a successful rename
and broke subsequent operations on its contents.

After a successful DirMove, poll the renamed folder's listing until it
shows the pre-rename child count, with a 30s timeout and exponential
backoff. The backend integration test already tolerated this via its
own list-consistency retries; this brings VFS into line.

Fixes #9450
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
ec28206d8a drime: remove duplicate upload_cutoff config option
The upload_cutoff option was registered twice with identical content.
Remove the second copy.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
ddbc886395 drime: fix files being uploaded to the wrong directory
Large files (sent as multipart uploads) were placed in the wrong folder
for two reasons:

- the parent folder was sent as "parent_id", but the API ignores that
  and expects "parentId", so the parent was never honoured
- relativePath was sent as the full path from the drive root, which made
  the server build folders from it and silently drop any "0" path
  segment (e.g. ".../data/0/file" lost the "0")

Send the parent as "parentId" and use just the leaf as relativePath,
matching the working single-part upload. This also lets us remove the
now-unneeded absolute-path resolution code.

Fixes #9392
Co-authored-by: Brian King <BrianDKing@gmail.com>
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
c8ceb209fc drime: fix server-side copy and move failing with Cloudflare 520 error
The drime origin returns a malformed response (reported by Cloudflare as
a 520 error) for a literal PUT request to the file-entries update
endpoint, which broke renaming, and so server-side copy and move.

Use a POST with the X-HTTP-Method-Override: PUT header instead - the API
routes this to the same handler and it works reliably.

Also retry Cloudflare 520-524 errors which may occur transiently.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
5b4eb0eefb serve sftp: fix truncate request being silently ignored
The SFTP serve handler ignored the size attribute of SETSTAT/FSETSTAT
requests, only acting on the modification time. This meant a client
asking to truncate a file (eg setting the final size of an upload, or
an explicit truncate) had no effect at all.

This respects the size attribute (if present) by truncating the file
to the requested size.
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
223e473e6e serve sftp: fix file corruption when a client resumes an upload
The SFTP serve write handler always opened files with O_TRUNC,
ignoring the flags requested in the SFTP OPEN packet. Some clients
(notably WinSCP's "Process in Background", which resumes an upload on
a second connection) re-open the partially written file without the
truncate flag and continue writing from the offset they had reached,
relying on the existing data being preserved. Forcing O_TRUNC zeroed
that prefix, so the start of the uploaded file ended up as a block of
zero bytes.

This fix respects the requested open flags instead so a resume open
without truncate keeps the already written data intact.

See: https://forum.rclone.org/t/rclone-serve-sftp-winscp-background-mode-uploading-causes-file-corruption/53841
2026-06-03 17:51:47 +01:00
Nick Craig-Wood
bac27547d5 Start v1.74.3-DEV development 2026-05-23 11:32:11 +01:00
Nick Craig-Wood
b22fe9811c Version v1.74.2 v1.74.2 2026-05-22 17:11:48 +01:00
Nick Craig-Wood
ce64f4edb8 gui: update embedded release to 1.1.8 2026-05-22 16:30:02 +01:00
Nick Craig-Wood
f3986e2bb7 lib/http: replace deprecated h2c.NewHandler with http.Server.Protocols
The golang.org/x/net/http2/h2c package was deprecated in v0.54.0 in
favour of setting the http.Server Protocols field to enable unencrypted
HTTP/2.

This replaces the h2c.NewHandler wrapping added in e863f751f with
http.Server.Protocols, which is supported by the standard library
since Go 1.24.

Note that the stdlib only supports HTTP/2 prior-knowledge on cleartext
connections, not HTTP/1.1 Upgrade: h2c negotiation. In practice clients
use prior-knowledge or require TLS, so this should not affect users.
2026-05-22 12:48:37 +01:00
Nick Craig-Wood
f0961f4a8d build: update golang.org/x dependencies to address multiple CVEs
Upgrade to v0.55.0 of golang.org/x/net in order to address:

- CVE-2026-42506: html: incorrect handling of namespaced elements in foreign content
- CVE-2026-39821: idna: failure to reject ASCII-only Punycode-encoded labels
- CVE-2026-42502: html: incorrect handling of HTML elements in foreign content
- CVE-2026-25680: html: denial of service when parsing arbitrary HTML
- CVE-2026-25681: html: incorrect handling of character references in DOCTYPE nodes
- CVE-2026-27136: html: duplicate attributes can cause XSS

Upgrade to version v0.52.0 of golang.org/x/crypto to address:

- CVE-2026-46598: ssh/agent: pathological inputs can lead to client panic
- CVE-2026-46597: ssh: byte arithmetic causes underflow and panic
- CVE-2026-39828: ssh: bypass of certificate restrictions
- CVE-2026-39835: ssh: server panic during CheckHostKey/Authenticate
- CVE-2026-39833: ssh/agent: key constraints not enforced
- CVE-2026-39832: ssh/agent: agent constraints dropped when forwarding keys
- CVE-2026-39827: ssh: memory leak when rejecting channels can lead to DoS
- CVE-2026-39830: ssh: client can cause server deadlock on unexpected responses
- CVE-2026-39829: ssh: pathological RSA/DSA parameters may cause DoS
- CVE-2026-39831: ssh: bypass of FIDO/U2F security keys physical interaction
- CVE-2026-39834: ssh: infinite loop on large channel writes
- CVE-2026-42508: ssh/knownhosts: auth bypass via unenforced @revoked status
- CVE-2026-46595: ssh: VerifiedPublicKeyCallback permissions skip enforcement

Upgrade to version v0.41.0 of golang.org/x/image to address:

- CVE-2026-42500: bmp: panic when reading out of bound palette index
- CVE-2026-33809: tiff: excessive resource consumption in PackBits decompression

Upgrade to version v0.45.0 of golang.org/x/sys to address:

- CVE-2026-39824: windows: integer overflow in NewNTUnicodeString
2026-05-22 12:48:37 +01:00
Iizuki
139ccc7ffd docs: s3: update doc link for force_path_style 2026-05-22 12:48:37 +01:00
Nick Craig-Wood
b3ddec50c6 s3: add new MEGA S4 endpoints on megas4.com including Asia-Pacific region
MEGA S4 has launched a new dedicated domain megas4.com with three new
endpoints, including their first Asia-Pacific location in Tokyo, plus
new endpoints in Paris and Barcelona. The existing Amsterdam,
Luxembourg, Montreal and Vancouver locations have been renamed onto
the new domain.

All current s4.mega.io endpoints continue to be supported and are
retained as legacy options. New accounts use the megas4.com endpoints.
2026-05-22 12:48:37 +01:00
Nick Craig-Wood
3cd224e03f docs: fix heading levels so sections appear in the table of contents
The "Import/Export of google documents" section in drive.md and the
"--b2-versions" examples section in b2.md were both at H4 instead of H3,
which excluded them from the ToC even though they are top-level topics
in their respective sections.
2026-05-22 12:48:37 +01:00
William Tange
52af9caa14 protondrive: fix corrupted on transfer: sha1 hashes differ
Some SHA1 hashes of protondrive files are uppercase, so always
converting to lowercase before comparing seems to be the solution.

Fixes #7345
2026-05-22 12:48:37 +01:00
Lucky945H
b2cb7f89c1 vfs/vfscache: fix silent write failure when mounting with remote:.
When mounting with 'remote:.' (dot notation for current directory),
the '.' is passed through toOSPath() which encodes it to full-width
'.' (U+FF0E) via EncodeDot. This caused the VFS cache data/meta root
to be written to a full-width-dot directory while the local cache
backend pointed to the canonicalized path, resulting in writes being
cached but never uploaded.

Fix by calling clean() on the relativeDirPath before toOSPath(),
so path.Clean() strips the trailing '/.' removing the dot before
encoding.
2026-05-22 12:48:37 +01:00
Leon Brocard
2292ba0cd4 docs: fix misspellings
Fix several misspellings in documentation and config comments detected by `misspell`.
2026-05-22 12:48:37 +01:00
Sai Asish Y
07c87327ae webdav: honour auth_redirect on listAll PROPFIND - fixes #9159 2026-05-22 12:48:37 +01:00
Tore Anderson
6cc0d1a0b0 jottacloud: support whitelabel service Phonero Sky
Phonero is a Norwegian mobile provider focusing on the enterprise
market. They provide rebranded Jottacloud service through their «Phonero
Sky» (Norwegian for «Phonero Cloud»).

The Client ID `desktop` used by most of the other whitelabel services
does not work for Phonero Sky. Both `desktop-win` and `desktop-mac`
work, though. Authentication does not work when the `jotta-default`
scope is present, so drop it. This results in a Client ID and scope
setup that is identical with that of Let's Go Cloud (Germany).
2026-05-22 12:48:37 +01:00
FTCHD
7a5e01b015 docs: add note about using config/update to reconnect an oauth remote 2026-05-22 12:48:37 +01:00
dependabot[bot]
bf111df84a build: bump github.com/go-git/go-billy/v5 to 5.9.0 to fix CVE-2026-44740 (again)
The previous commit had no content due to copilot deleting it as I was
merging it!

Bumps [github.com/go-git/go-billy/v5](https://github.com/go-git/go-billy) from 5.8.0 to 5.9.0.
- [Release notes](https://github.com/go-git/go-billy/releases)
- [Commits](https://github.com/go-git/go-billy/compare/v5.8.0...v5.9.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-billy/v5
  dependency-version: 5.9.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-22 12:48:37 +01:00
Nick Craig-Wood
0b73e2b858 docs: upgrade protondrive to tier 4 tests pass and community maintains 2026-05-22 12:48:37 +01:00
Alvinwylim
e0f210cdc4 drime: fix 500 errors when listing shared folders
When using the drime backend to access a folder shared from another
Drime account (via root_folder_id config option), listing the folder's
children fails with HTTP 500. This makes cross-account read-only
mounts non-functional.

This fixes the problem by changing listAll to use `folderId` instead
of `parentIds[]`

Fixes #9420
2026-05-22 12:48:37 +01:00
nielash
1c06bb9272 bisync: fix --conflict-loser pathname with --conflict-resolve newer
Before this change, --conflict-loser pathname assumed --conflict-resolve none,
following the legacy behavior prior to v1.66. This produced unexpected behavior
when used with a different --conflict-resolve option.

This change fixes the issue by ensuring that --conflict-loser pathname looks for
the correct name on the side not being renamed, when only one side should be
renamed.

https://forum.rclone.org/t/bisync-does-not-copy-the-winner-file-to-the-loser-site/53768
2026-05-22 12:48:37 +01:00
Nick Craig-Wood
b2e1aa65dc rc: remove duplicate metrics_addr option registration
The metrics_addr option was registered twice: once explicitly and once
implicitly via AddPrefix(libhttp.ConfigInfo, "metrics", ...). Both
pointed at the same MetricsHTTP.ListenAddr field, so options/info
returned a duplicate entry.

Drop the explicit entry and use SetDefault to keep the empty default
(so the metrics server stays off unless configured), matching the
pattern already used for rc_addr.

Fixes #9419
2026-05-22 12:48:37 +01:00
John Volk
3a414d8262 drime: fix file doesn't exists error when trying to delete
When using rcat to upload a new version of a file that already existed,
the file upload would succeed. The subsequent deletion of the old file
is attempted after the upload. Drime appears to handle the deletion of
the old file automatically and returns HTTP status code 422, stating
the "The selected entry ids is invalid."

The deletion and the rcat would fail before this change. This is with
file history enabled on my Drime account.

This change detects the error and ignores it since the file has
already been deleted.
2026-05-22 12:48:37 +01:00
Nick Craig-Wood
24054305ce Start v1.74.2-DEV development 2026-05-08 17:14:19 +01:00
Nick Craig-Wood
076fb2bc53 Version v1.74.1 v1.74.1 2026-05-08 15:15:36 +01:00
Nick Craig-Wood
e1fd3725bc build: update golang.org/x/net to v0.53.0 to fix CVE-2026-33814 2026-05-08 15:07:27 +01:00
Nick Craig-Wood
5780a30c30 build: fix multiple CVEs by upgrading to go1.26.3
- CVE-2026-42501: cmd/go: malicious module proxy can bypass checksum database
- CVE-2026-39825: net/http/httputil: ReverseProxy forwards queries with more than urlmaxqueryparams parameters
- CVE-2026-39836: net: panic in Dial and LookupPort when handling NUL byte on Windows
- CVE-2026-42499: net/mail: quadratic string concatenation in consumePhrase
- CVE-2026-39820: net/mail: quadratic string concatentation in consumeComment
- CVE-2026-39819: cmd/go: "go bug" follows symlinks in predictable temporary filenames
- CVE-2026-39817: cmd/go: "go tool pack" does not sanitize output paths
- CVE-2026-33814: net/http: infinite loop in HTTP/2 transport when given bad SETTINGS_MAX_FRAME_SIZE
- CVE-2026-39826: html/template: escaper bypass leads to XSS
- CVE-2026-33811: net: crash when handling long CNAME response
- CVE-2026-39823: html/template: bypass of meta content URL escaping causes XSS
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
17c2db7289 drime: fix uploads of 100..200M files
At some point Drime recommended 200M for the upload cutoff for
switching to multipart upload. However uploads have stopped working
using single part upload for 100..200Mish files.

Their docs now recommend 5M as the cutoff for multipart upload so this
changes the default.
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
44c56ed481 drime: fix large file uploads landing in drive root instead of configured folder
The /s3/multipart/create and /s3/entries endpoints interpret relativePath
as an absolute path from the drive root, not relative to parent_id. When
root_folder_id was set to a non-root folder, files larger than
upload_cutoff ended up at the user's drive root instead of the configured
folder.

Resolve the absolute path of the Fs root once via GET /folders/{hash}/path
(cached on first OpenChunkWriter call) and use that to build the correct
relativePath.

Fixes #9392
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
ebca82f7db docs: sponsor updates
- Add Data Raven as a silver sponsor
- Add Impossible Cloud as a bronze sponsor
- Shuffle silver sponsors once per page load
- Remove TOC from sponsors page
2026-05-08 15:07:27 +01:00
Leon Brocard
e9d280b793 s3: add new Fastly Object Storage regions
Add three new regions and their endpoints for Fastly Object Storage:

- eu-west-1 (Paris)
- us-east-1 (Virginia)
- us-west-1 (Oregon)

These are distinct from the existing us-east, us-west and eu-central
endpoints, which are kept in place.
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
4f93bd482f cloudinary: fix retrying every error and fix pacer sleep units
shouldRetry treated every non-nil error as retryable, so permanent
failures (auth, 4xx, not-found) burned through the LowLevelRetries
budget instead of returning fast.

This also fixes the pacer sleeps: pacer.MinSleep(1000) and
MaxSleep(10000) are time.Duration values, so they were 1µs and 10µs -
almost certainly intended as 10ms and 2s.
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
b6c68882e5 s3: fix STS call per request by caching AssumeRole credentials
The stscreds.AssumeRoleProvider from AWS SDK Go v2 does not cache
credentials by itself. The SDK only auto-wraps providers with
aws.CredentialsCache when they are loaded via
config.LoadDefaultConfig; when assigned directly to
aws.Config.Credentials it must be wrapped manually, as documented on
stscreds.NewAssumeRoleProvider.

Without the cache, configurations using role_arn would call AssumeRole
once per S3 request, flooding STS and CloudTrail.

See: https://forum.rclone.org/t/aws-iam-roles-credentials-arent-cached/53732
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
23dc9d0515 protondrive: fix segfault when copying files missing revision metadata
When a Proton Drive file has no active revision attributes,
readMetaDataForLink returns a nil FileSystemAttrs and Object.originalSize
is left as nil. Object.Open then dereferenced this nil pointer when
calling fs.FixRangeOption, causing a SIGSEGV during copy.

Use Object.Size() instead, which already implements the correct fallback
to the link size when originalSize is unavailable.

This updates the github.com/rclone/Proton-API-Bridge package to fix a
segfault when reading files with no metadata.

Fixes #9377
Fixes #9117
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
be7c0f31a4 protondrive: route library logging through rclone's logger
Previously all log output produced by Proton-API-Bridge (stdlib log)
and go-proton-api (logrus + resty's logger) bypassed rclone's
logging: it ignored -v / -vv levels and didn't reach --log-file.

Add a small adapter implementing the resty.Logger / bridge Logger
shape that calls fs.Errorf / fs.Logf / fs.Debugf, and pass it via
the new Config.Logger hook. The bridge in turn forwards the same
value to go-proton-api's WithLogger option, so HTTP-layer warnings
and the formerly-hardcoded logrus warnings inside go-proton-api
also surface through rclone's log levels.
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
3ac9b75a83 protondrive: route HTTP through rclone's transport
The Proton Drive backend constructed the upstream Proton-API-Bridge
without ever passing rclone's HTTP transport. As a result none of
rclone's HTTP flags reached Proton: --dump headers, --dump bodies,
--no-check-certificate, --user-agent, --bind, --ca-cert, --header,
--tpslimit etc. all silently did nothing for this remote, and HTTP
traffic was invisible to -vv.

Pass fshttp.NewTransport(ctx) through the new Config.Transport hook on
the bridge, which forwards it to the updated go-proton-api's
WithTransport option and so to the underlying resty client.
2026-05-08 15:07:27 +01:00
Gustavo V. F.
5e9cb39084 bisync: fix retryable without --resync error message when --resync has a critical failure 2026-05-08 15:07:27 +01:00
Leon Brocard
87d0b59a51 cmd/serve/s3: return object listings in key order
The S3 ListObjects response from `rclone serve s3` was sorting object
contents by modification time instead of object key. This made the
listing order incompatible with S3 clients which expect lexicographic
key ordering.

In particular, `aws s3 sync` assumes both source and destination
iterators are ordered by key. With the old modtime ordering it could
misidentify files as missing or outdated and re-download objects that
were already up to date.

Change the pager to sort returned objects by key and add a regression
test which uses keys and modtimes arranged so the old behaviour would
fail.

Fixes #9002
2026-05-08 15:07:27 +01:00
Nick Craig-Wood
76596b6727 Start v1.74.1-DEV development 2026-05-08 15:07:27 +01:00
Nick Craig-Wood
aa031c51cc Version v1.74.0 v1.74.0 2026-05-01 15:56:56 +01:00