This commit allows global config options to be set as flat parameters on all rc
commands. This makes the rc commands much more like command line parameters and
will aid understanding of how the rc is used.
It is backwards compatible with the old method using _config and _filter.
A connection string can carry global.* options which change rclone's
process-wide configuration (e.g. global.http_proxy). This is
undesirable for the rc interface which was designed to have multiple
users or connections at once. The rc interface has the `_config`
mechanism for setting request scoped global config.
This blocks global.* options on all rc paths by marking the context as
a remote control request at the rc boundaries. fs.NewFs then skips
applying global.* to the process-wide config for a marked context.
The marker is reapplied in fs.CopyConfig, which is the call rclone
uses to detach context but keep config.
global.* options still apply to the individual backend they are set
on, exactly like override.* options; they just no longer leak into the
rest of the process. Remotes created directly on the command line are
unaffected as are remotes defined in the config file.
See: GHSA-qw24-gh76-8rvv
The --rc-serve GET/HEAD file serving path accepted bracketed inline
remotes from the URL and instantiated them, so a single
unauthenticated request could run a command as the rclone user via
backend options such as webdav bearer_token_command or sftp ssh, read
arbitrary local files, or change process-wide config via global.*
options.
This was the GET/HEAD equivalent of the POST hole fixed for
CVE-2026-41179, which only guarded the rc call dispatch path.
Now, unless the rc server has authentication configured or
--rc-no-auth is set, the serve path only allows remotes already
present in the config file: inline remotes, connection string
parameters and bare local paths are rejected. Connection string
global.* options are never honoured on the serve path, even when
authenticated.
See: GHSA-qw24-gh76-8rvv
Make rc respond with a 202 status code (instead of 200) if `Prefer: respond-
async` was passed. Keeps backwards compatibility for current clients while also
allowing the OpenAPI schema & generators to differentiate the responses
properly.
The metrics_addr option was registered twice: once explicitly and once
implicitly via AddPrefix(libhttp.ConfigInfo, "metrics", ...). Both
pointed at the same MetricsHTTP.ListenAddr field, so options/info
returned a duplicate entry.
Drop the explicit entry and use SetDefault to keep the empty default
(so the metrics server stays off unless configured), matching the
pattern already used for rc_addr.
Fixes#9419
Replace AuthRequired bool with NoAuth bool on the rc.Call struct and
flip the auth check logic. Previously endpoints were unauthenticated
by default and had to opt in with AuthRequired: true, which led to
security vulnerabilities when developers forgot to set the flag.
Now all endpoints require authentication by default. Only explicitly
safe read-only endpoints are marked with NoAuth: true:
- rc/noop
- rc/error
- rc/list
- core/version
- core/stats
- core/group-list
- core/transferred
- core/du
- cache/stats
- vfs/list
- vfs/stats
- vfs/queue
- job/status
- job/list
See GHSA-25qr-6mpr-f7qx, GHSA-jfwf-28xr-xw6q
Snapshot the NoAuth setting when the RC server is created rather than
reading it from the mutable options struct on each request. This
prevents any runtime mutation of rc.NoAuth (e.g. via options/set)
from disabling the auth gate for protected RC methods.
See GHSA-25qr-6mpr-f7qx
The options/set RC endpoint was registered without AuthRequired,
allowing unauthenticated callers to mutate global runtime options
including rc.NoAuth, which disables the auth gate for all protected
RC methods. Require authentication for options/set.
See GHSA-25qr-6mpr-f7qx
Bind the RC server to localhost:0 and read the bound URL back via a
new rcserver.Server.URLs() accessor instead of pre-allocating a port
in cmd/gui. This removes the small TOCTOU race window between
freePort() closing its listener and rcserver claiming the same port.
Before this change, if any code called fs.Fatal(f) then it would stop
rclone as designed. However this is not appropriate when using the RC
API - we want the error returned to the user.
This change turns the fs.Fatal(f) call into a panic which is caught by
the RC API handler and returned to the user as a 500 error.
All user visible Durations should be fs.Duration rather than time.Duration. Suffix is then optional and defaults to s. Additional suffices d, w, M and y are supported, in addition to ms, s, m and h - which are the only ones supported by time.Duration. Absolute times can also be specified, and will be interpreted as duration relative to now.
This was caused by an incorrect handler URL which was passing the
debug/* commands to the debug/pprof handler by accident. This only
happened when using unix sockets.
Some libraries use `application/json; charset=utf-8` as their `Content-Type`, which is valid.
However we were not decoding the JSON body in that case, resulting in issues communicating with the rcserver.
This commit modernizes Go usage. This was done with:
go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...
Then files needed to be `go fmt`ed and a few comments needed to be
restored.
The modernizations include replacing
- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
Fix in extreme cases, when the job is executing finish(), the listener added by calling OnFinish() will never be executed.
This change should not cause compatibility issues, as consumers should not make assumptions about whether listeners will be run in a new goroutine
Before this change, when cache.GetFn was called on a file rather than a
directory, two cache entries would be added (the file + its parent) but only one
of them would get pinned if the caller then called Pin(f). This left the other
one exposed to expiration if the ci.FsCacheExpireDuration was reached. This was
problematic because both entries point to the same Fs, and if one entry expires
while the other is pinned, the Shutdown method gets erroneously called on an Fs
that is still in use.
An example of the problem showed up in the Hasher backend, which uses the
Shutdown method to stop the bolt db used to store hashes. If a command was run
on a Hasher file (ex. `rclone md5sum --download hasher:somelargefile.zip`) and
hashing the file took longer than the --fs-cache-expire-duration (5m by default), the
bolt db was stopped before the hashing operation completed, resulting in an
error.
This change fixes the issue by ensuring that:
1. only one entry is added to the cache (the file's parent, not the file).
2. future lookups correctly find the entry regardless of whether they are called
with the parent name or one of its children.
3. fs.ErrorIsFile is returned when (and only when) fsString points to a file
(preserving the fix from 8d5bc7f28b).
Note that f.Root() should always point to the parent dir as of c69eb84573
This changes log statements from log to fs package, which is required for --use-json-log
to properly make log output in JSON format. The recently added custom linting rule,
handled by ruleguard via gocritic via golangci-lint, warns about these and suggests
the alternative. Fixing was therefore basically running "golangci-lint run --fix",
although some manual fixup of mainly imports are necessary following that.
There were a lot of instances of this lint error
printf: non-constant format string in call to github.com/rclone/rclone/fs.Logf (govet)
Which were fixed by re-arranging the arguments and adding "%s".
There were quite a few genuine bugs which were found too.
This adds an additional parameter to the creation of each flag. This
specifies one or more flag groups. This **must** be set for global
flags and **must not** be set for local flags.
This causes flags.md to be built with sections to aid comprehension
and it causes the documentation pages for each command (and the
`--help`) to be built showing the flags groups as specified in the
`groups` annotation on the command.
See: https://forum.rclone.org/t/make-docs-for-mortals-not-only-rclone-gurus/39476/
