The go-smb2 dependency's vfs/fileinfo_linux.go assumes 64-bit
syscall.Stat_t time fields which are int32 on 32-bit architectures.
This excludes linux/386, linux/arm, linux/mips and linux/mipsle
from building serve smb.
Add a new `rclone serve smb` command that serves any rclone remote
over the SMB protocol using the github.com/macos-fuse-t/go-smb2
server library.
Fixes#7596
Some S3-compatible servers (e.g. Archiware P5) reject requests with an
empty `?delimiter=` query parameter. For recursive listings, pass `nil`
instead of a pointer to an empty string so the parameter is omitted
entirely from the request.
Fixes#9342
The temp directory name used random.String(2) giving only 676 possible
values. When multiple concurrent tests started in the same second, they
shared the same timestamp prefix, causing name collisions and shared
temp directories. This led to lock file conflicts, listing file races,
and file deletion errors.
Increase to random.String(8) to make collisions effectively impossible.
Before this change, if an object compressed with "Content-Encoding:
gzip" was downloaded, a length and hash mismatch would occur since the
go runtime automatically decompressed the object on download.
If --azureblob-decompress is set, this change erases the length and hash on
compressed objects so they can be downloaded successfully, at the cost
of not being able to check the length or the hash of the downloaded
object.
If --azureblob-decompress is not set the compressed files will be downloaded
as-is providing compressed objects with intact size and hash
information.
Fixes#9337
Add a Hugo page at /backends/index.json that exports all the
backend YAML data from docs/data/backends/ as a single JSON file
for external consumption.
Set the User-Agent to include the APN prefix for Azure backends
(azureblob, azurefiles, onelake) to identify rclone as a Microsoft
Partner. The User-Agent is now:
APN/1.0 rclone/1.0 rclone/<version>
Add a ctx parameter to vfs.New() so callers can pass in context
carrying ConfigInfo and FilterInfo. The context is stripped of
cancellation but config and filter values are preserved into a fresh
background context.
Add a ctx field to the VFS struct, initialized in New() from the
existing cancellable context. Propagate this through the cache
subsystem hierarchy.
This ensures proper context cancellation when a VFS shuts down, rather
than using disconnected context.TODO() or context.Background() calls
throughout and paves the way for VFS to have its own config.
The flag.Lookup("test.v") check existed to skip opening a browser
during tests, but the tests don't exercise RunE, so this was never
used. The --no-open-browser flag is sufficient on its own.
Bind the RC server to localhost:0 and read the bound URL back via a
new rcserver.Server.URLs() accessor instead of pre-allocating a port
in cmd/gui. This removes the small TOCTOU race window between
freePort() closing its listener and rcserver claiming the same port.
- Fail gracefully if `make fetch-gui` hasn't been run
- Return errors instead of panic or fatal errrors
- Don't run `make fetch-gui` on every make since we have it in the workflow
The fetch-gui-dist.sh script calls the GitHub releases API
unauthenticated, which is limited to 60 requests/hour per source IP.
GitHub Actions runners share outbound IPs, so this quota is regularly
exhausted.
Pass GITHUB_TOKEN (or GH_TOKEN) as an Authorization header when
present, raising the limit to 1000/hour, and wire secrets.GITHUB_TOKEN
into the workflow step. Local unauthenticated runs still work.
`json:"entry_permissions"` is known to be either empty [] or of
structure {string: boolean}. This may have been a breaking API change on
Drime's side. Because EntryPermissions is not used, the type was changed
to `any` to capture both cases, otherwise we could implement custom
unmarshalling for that type.
These Debugf calls in NewFilter() ran during GlobalOptionsInit(), before
InitLogging() configured the JSON log format. This caused plain-text
debug lines to leak to stderr when --use-json-log was set, breaking
tooling that expected only JSON output.
The resolved time values are already available via --dump filters so
this commit removes the debug messages.
This adds a new gui command which runs an embedded copy of the GUI at
https://github.com/rclone/rclone-web/
The GUI release is fetched as part of the CI build.
The Global Acceleration Endpoint (cos.accelerate.myqcloud.com) of
Tencent COS does not seem to support "CreateBucket" (maybe also other
bucket management operations). Since the acceleration functionality must
be enabled per-bucket in the Tencent Cloud console, the bucket will
always exist before this endpoint is used, so this check can be safely
skipped.
Now, "no_check_bucket = true" will be auto set when using this endpoint.
Why "NewFs()": on-the-fly remotes (connection string remotes), for
example, ":s3,provider=TencentCOS,...:..." will also be fixed.
Why no unit test: I can't find a good way to test "NewFs()" without
leveraging live endpoints. I think we can extract all existing mutations
for different providers (e.g., AWS, Fastly, and Rabata) from "NewFs()"
to a new function in the future.
Some Tencent docs about this CDN endpoint:
- English: Global Acceleration Endpoint | https://www.tencentcloud.com/pt/document/product/436/40700
- Chinese: 对象存储 全球加速概述_腾讯云 | https://cloud.tencent.com/document/product/436/38866
Assisted-By: OpenCode
When a user has --s3-versions set but lacks the s3:GetBucketVersioning
permission, GetBucketVersioning returns an error and isVersioned() caches
the result as false. This caused CleanUpHidden (backend cleanup-hidden) to
silently exit with "bucket is not versioned so not removing old versions",
ignoring the user's explicit --s3-versions flag.
Fix this by trusting the explicit --s3-versions flag in purge(), bypassing
the GetBucketVersioning check when the user has explicitly declared the
bucket is versioned.
Previously ssh.InsecureIgnoreHostKey() was set unconditionally as the
default HostKeyCallback with no indication to the user.
This logs a warning pointing users to the documentation on how to
enable host key validation.
See: https://github.com/rclone/rclone/security/code-scanning/167
Before this change when a cache item was in its grace period (with
HandleCaching) and the file is reopened, _checkObject runs before the
grace timer recovery check. If the remote object's fingerprint changed
_checkObject removes the cache file from disk. However the grace
recovery path still reused the now-stale fd pointing to a deleted
inode, skipping _createFile entirely. This left no cache file on disk,
causing cache.Exists() to return false and breaking
rename-while-writing logic.
Fix this by checking the cache file still exists before reusing the fd
in grace recovery. If the file was removed, close the stale fd and
downloaders and fall through to _createFile.
Also update the fingerprint in item.rename after setting the new object,
preventing unnecessary cache invalidation when a file is reopened after
a rename.
This was discovered in the integration tests on backends that update
modtime on rename (like mailru).
The Linkbox open API (/api/open/file_search) no longer returns download
URLs, breaking all downloads. This switches to using the web API
(/api/file/my_file_list/web) which requires email+password authentication
but returns working download URLs.
This will unfortunately require changing your existing rclone config.
- Add email, password, and web_token config options
- Add web API login via /api/user/login_email with token caching and retry
- Create separate CDN HTTP client with HTTP/2 disabled and browser
User-Agent to avoid CDN fingerprint blocking
- Remove searchOK regex and name-filtering (web API doesn't support it)
In this commit
0db3e7a2a0 vfs: fix slow nfs serve by adding --vfs-handle-caching
We added --vfs-handle-caching but unfortunately forgot to disable it
for the TestRWCacheUpdate test.
Add a configurable grace period (default 5s) that delays closing file
handles and downloaders when the last handle closes. If a new handle
opens within the grace period, it reuses the existing resources.
This fixes 40x performance degradation with serve nfs vs serve sftp
caused by go-nfs opening/reading/closing on every NFS READ RPC, which
destroyed read-ahead prefetch before it could accumulate.
The grace period only applies to non-dirty files so that writeback
proceeds immediately on close.
Fixes#9251
