Before this change setting LogLevel in the _config parameter of an rc
call had no effect as both the log functions and the log handler only
looked at the global log level.
The context aware log functions now use the log level from the config
in the context, and the log handler lets a log level in the context
which differs from the global one override its own level. This means
an rc call can be made more or less verbose than the rest of rclone
with, for example
"_config": {"LogLevel": "DEBUG"}
This applies to the logs made with the context aware log functions, so
not to those made by the backends which still use the global log level.
Note that a config copied into a context no longer follows later
changes to the global log level.
Several links on rclone.org pointed at anchors or pages that don't exist:
- bisync: --color and --dry-run anchors, relative commands/rclone_check/ links
- ftp: on-the-fly link pointed at a local anchor instead of /docs/
- faq: --sftp-http-proxy linked to the ftp page instead of sftp
- index: Arvan Cloud and Liara provider config anchors
- rc, sftp: Hugo drops leading dashes from heading ids (#--rc-no-auth -> #rc-no-auth)
- onedrive: webdav #sharepoint -> #sharepoint-online
- webdav: relative commands/rclone_serve_webdav/ link
- s3: https:/// typo in the China Mobile Ecloud link
Before this change, when an existing file had changed on both paths, the
conflict winner was chosen using only the attributes that had changed on each
side. An attribute that hadn't changed was treated as zero, so --conflict-resolve
larger could pick the smaller file (for example, when one side only had its
modtime updated), and --conflict-resolve newer could fail to pick a winner when one
side's modtime hadn't changed.
After this change, the current size, modtime, and checksum (to the extent
enabled with compare settings) are now recorded for every changed file.
Before this change, when --conflict-resolve picked Path2 as the winner, the winning
file was copied to Path1 correctly but then removed from both listings while
updating them for the renamed (or deleted) Path1 copy. The next run saw the file
as new on both paths and had to check it again, and a change to it on one path
in that run was handled as if the file had never been synced. When Path1 won,
the listings happened to be updated in an order that avoided this.
This change fixes the issue by adding a guard to remove the file from the
listing only when dstNewName != dstOldName.
See #9939
The manual release verification steps checked the signature with gpg
--verify and then checked the binaries against the whole SHA256SUMS
file. gpg only verifies the clearsigned part, so an unsigned hash line
added outside it would be used by sha256sum -c or by rclone hashsum -C.
The instructions now extract the signed hashes with gpg --decrypt and
check the binaries against those only.
See: GHSA-w33h-3qgq-8mv8
The transport strips the headers set with --header once a redirect
chain leaves the originally requested host, but the host comparison
ignores the URL scheme. A server which redirects https://host:8443/
to http://host:8443/ keeps the same host and port, so the headers
were added to the plaintext request and could be read by anyone on
the network path.
The redirect chain walk now also treats a hop which downgrades the
original HTTPS request to HTTP as leaving the host, so the headers are
removed from that hop and every one after it.
The transport also logs at debug level when it removes the headers,
for either reason, so that a resulting 401 from the redirect target
can be diagnosed.
The http backend resolves the path after `remote:` against the
configured url according to the URL standard. A path beginning with
`//` is a network-path reference under that standard, so a path such as
`//other.example/dir/` replaced the configured host (and port and
userinfo) rather than being treated as a path. The configured headers
were then sent to the substituted host on the initial HEAD request and
every request after it.
Use rest.URLJoinRoot, which refuses any root carrying a scheme, host
or userinfo of its own, to join the path onto the configured url.
Before this change the basic, htpasswd and auth proxy authentication
middlewares let every OPTIONS request through without checking
credentials so that browser CORS preflights, which can't carry
credentials, would succeed.
However in serve webdav the OPTIONS request is answered by the WebDAV
handler whose Allow header depends on whether the path is an existing
file, an existing directory or doesn't exist, so an unauthenticated
client could discover the existence and type of any path on the served
remote even with --user/--pass or --htpasswd set.
After this change he authentication middlewares no longer special-case
OPTIONS. Instead the CORS middleware, which runs before authentication
and owns the --allow-origin setting, answers genuine CORS preflights
itself with an empty 200 carrying the CORS headers only when
--allow-origin is set.
All other OPTIONS requests need credentials like any other method, so
clients which probe with OPTIONS before authenticating will get a 401
challenge and must retry with credentials.
A symlink in an archive was listed as an ordinary file holding the link
target by the zip backend, and dropped entirely by the squashfs one.
Symlinks are now skipped unless -l/--links is in use, as they are on the
other backends. With that flag they are listed with the .rclonelink
suffix and read back as the path they point at, so they can be copied to
a destination which supports symlinks.
This changes what the zip backend lists by default: a symlink used to
appear as a small file whose contents were the link target, and is now
skipped with a NOTICE naming the flag.
Fixes#9569
Add a new backend for dosya.dev, a cloud file storage, sharing, and
synchronization platform with workspace-based multi-tenancy.
Features:
- Server-side copy, move, rename (files and directories)
- Purge (recursive directory delete)
- Recursive listing (ListR) for --fast-list
- Multipart upload for large files (>10MB)
- Public link sharing via rclone link
- About (workspace storage quota)
- MIME type detection
- Empty directory support
- Workspace-based multi-tenancy (one remote per workspace)
- Download via presigned R2 URLs with Range header support
Co-authored-by: firatkaya <firat@netiket.com.tr>
Files.com now treats the replacement characters rclone uses for / and
\ in file names (／ and ＼) as if they were the originals, and the
names ． and ．． as if they were . and .., and rejects them all with
"Invalid path". There is no other way of encoding these names, so
document the restriction and ignore the integration tests for them.
With -l/--links a .rclonelink object is buffered in memory to become
the target of a symlink. The read was unbounded, so a hostile or
corrupt source serving a large .rclonelink object made rclone use that
much memory and then log the whole body in the resulting error.
A symlink target can never be longer than a path, so the read now stops
at 128 KiB and anything longer is refused without retrying.
The docs describe a duration as a sequence of numbers each with a unit
suffix, including d, w, M and y, but a sequence that used one of those
units, such as --max-age 1d12h, failed with a confusing error about
parsing it as a date. Only a single number with one of them, like 1.5d,
or a sequence of the time.ParseDuration units was accepted.
Parse such sequences before falling back to dates, rejecting ones too
long to represent.
Add an opt-in preflight that excludes only guaranteed tracked renames
from the max-delete count. Reuse normal bisync listing metadata,
validate strategy and flag conflicts early, and skip preflight work
when --force bypasses the safety check.
The max_delete_track_renames scenario uses the default hash
track-renames strategy, which requires a hash common to both paths.
Skip it on remote combinations without one, such as crypt, instead of
failing the integration tests.
Fixes#8685
Reading an object in an archive storage class failed with "Object in
GLACIER, restore first" even when the object was in DEEP_ARCHIVE or in
an Intelligent-Tiering archive access tier.
Use the storage class and access tier from the InvalidObjectState
error. The storage class is optional in that error, so fall back to
the storage class from the listing or HEAD, and then to GLACIER as
before.
Rclone always asked for the delta listing of the whole drive and threw
away the items which weren't under the directory being listed.
The delta API now works on any folder, so ask for the delta listing of
the directory being listed instead, falling back to listing from the
root of the drive for drives which only support delta there.
Shortcuts to shared items (and Personal Vault) are stored as remote
items pointing at another drive and listing them can fail with
"The provided drive id appears to be malformed". This aborts a
--fast-list listing of the whole drive.
Testing against OneDrive personal (free) and OneDrive for Business shows
- personal accounts now create versions on setting the modification
time and can delete them, so --onedrive-no-versions and rclone
cleanup work there
- --onedrive-link-password works on OneDrive for Business
- personal free accounts can't set a link password or --expire
- --onedrive-link-type embed only works on OneDrive personal
- personal accounts store times with 1s precision, not mS
See #9917
With -L/--copy-links a symlink pointing to one of its parent
directories was followed until the OS gave up with "too many levels
of symbolic links" 40 levels deep. As every looping symlink multiplies
the listing, a small tree with three such symlinks listed millions of
entries.
When a followed symlink points to a directory, compare it with the
directory being listed and each of its parents using os.SameFile and
if it matches report an error and skip it, the same way as a circular
symlink. A symlink to a sibling directory is still followed. Loops
excluded by the directory filters are skipped quietly as before.
The link pointed at a bare relative path (--check-filename) instead of
the in-page anchor for the ### --check-filename heading further down the
page, so it 404s on the rendered docs site. Point it at #check-filename
(Hugo strips leading dashes when generating header anchors).
---------
Co-authored-by: no-hup <19599684+no-hup@users.noreply.github.com>
Ceph RGW (and Linode Object Storage, which is Ceph-backed) can break
SigV4 when Accept-Encoding is included in the signature, especially
when a reverse proxy rewrites that header. GCS already sets this quirk;
apply the same default for Ceph and Linode as suggested in #8206.
Google now requires an app homepage URL and privacy policy URL to be
set on the OAuth consent screen's "Branding" page before the "PUBLISH
APP" button becomes clickable, even for a personal single-user app.
The existing instructions jumped straight to publishing in step 9
without mentioning this, leaving the button greyed out with no
explanation of why.
Fixes#9854
Nick Craig-Wood