Return the first non-EOF input error encountered while Rcat probes whether an
upload is small. Previously that error was ignored and the full probe buffer,
including bytes that were never read, could be passed to Put or PutStream.
- Reject uploads when object lookup fails for reasons other than "not found".
- Serialize uploads to the same object to prevent races between the
existence check and the write.
Gzip metadata is read from the wrapped remote and could contain an
invalid block size or incomplete block index. A range read could then
panic in the seekable gzip reader.
Validate the gzip sidecar invariants before constructing a reader so
malformed remote metadata returns an error instead of crashing rclone.
The start function was run inside an if condition, where bash ignores
errexit, so a failed docker run was not noticed. The script then
printed its connection details anyway and the test spent 100 seconds
trying to connect before failing with a message that hid the real
error.
Run start in a subshell with errexit on and check its status
explicitly so the failure is reported immediately with docker's
error message.
The minio/minio repository has been removed from Docker Hub so the
TestS3Minio test server could not be started and every Linux CI run
failed. quay.io/minio/minio still serves the final release so use
that instead.
OCI rejects a multipart completion request with no parts. Probe unknown-size
streams through a buffered reader and use a regular upload when the stream is
empty. Peek preserves a non-empty stream for the selected upload path.
Internxt stores a file as a (plainName, type) pair and never derives the
split itself, so it is a convention shared between clients. This backend
split at the final dot, storing and looking up ".bashrc" as an empty name
of type "bashrc", where the web, desktop, Linux and macOS clients all keep
the leading dot in plainName. List rebuilt the full name so such files
appeared, but NewObject looked them up by the split and missed them.
- Replaced the preUploadCheck function with findFile for better clarity and efficiency in checking file existence.
- Introduced splitNameExt to encapsulate name and extension parsing logic.
- Updated NewObject and Update methods to utilize findFile for improved file metadata handling.
- Enhanced error handling and reduced redundant code in file checks.
Use OCI BatchDeleteObjects to remove up to 1,000 objects per request when
purging. Include directory markers so purging a bucket leaves it empty for
deletion.
OCI Object Storage rejects NUL, carriage return and line feed in
object names. NUL is already encoded by rclone, while carriage returns
and line feeds need EncodeCrLf so rclone can represent those names
without issuing invalid OCI requests.
https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/managingobjects.htm
When rclone archive create is run without a destination the archive
should be written to stdout, but ArchiveCreate called
CheckValidDestination on the nil dst and panicked with a nil pointer
dereference. Skip the destination check when there is no destination.
Also remove a leftover debug Printf that wrote to stdout before the
archive data.
Signed-off-by: Vladimir Babin <vovababin@gmail.com>
Use OCI RenameObject for moves within a bucket, avoiding a server-side copy
followed by source deletion. Cross-bucket moves continue to fall back to copy
and delete.
The link pointed at a bare relative path (--check-filename) instead of
the in-page anchor for the ### --check-filename heading further down the
page, so it 404s on the rendered docs site. Point it at #check-filename
(Hugo strips leading dashes when generating header anchors).
---------
Co-authored-by: no-hup <19599684+no-hup@users.noreply.github.com>
Before this --onedrive-upload-cutoff was capped at 4 MiB and made
larger single-part uploads impossible.
Microsoft documents the limit for a single-request upload as "250 MB".
Measured against SharePoint Online the figure is binary and exclusive:
a body of 262143999 bytes is accepted and one of 262144000 is not.
Raise the constant to 250 MiB and record where it comes from.
The default is unchanged (upload_cutoff = -1, always chunk), so this
only affects users who raise the cutoff deliberately.
Co-authored-by: Can Arslan <carslan@viyaenv.com>
Listing a bucket without -R shows each prefix as one directory entry, so the
objects under it are not in the output. This is easy to read as a truncated
listing rather than as one level of a hierarchy.
Fixes#9797
A vulnerability exists in gRPC-Go servers configured with
xds.NewGRPCServer() where a crafted request missing both :authority
and Host headers can cause a server panic, resulting in a Denial of
Service (DoS).
This update fixes the problem.
NewUsageValue exists to clip an oversized quota to the maximum value of an
int64, which is what dc95f36bc added it for when Box raised the Enterprise
space_amount to 1e+18 and started returning it as a float.
For the float64 instantiation the guard misses its own boundary.
float64(math.MaxInt64) is not 2**63-1, it rounds up to 2**63, so a quota of
exactly 2**63 fails the comparison and falls through to the int64 conversion,
which the spec leaves implementation dependent for an unrepresentable value.
On linux/amd64 it wraps:
Before: rclone about -> Total=-9223372036854775808
After: rclone about -> Total=9223372036854775807
A negative total is not just a wrong number. vfs.Statfs documents -1 as "not
known", vfs.fillInMissingSizes branches on total < 0, and serve sftp only
computes its usage percentage when total > 0, so the value is read back as a
missing quota.
The int64 and uint64 instantiations are unaffected, since for them
T(int64(math.MaxInt64)) is exact and clipping MaxInt64 to MaxInt64 is a no-op.
Set built the timetable with *x = append(*x, ts), so setting a bandwidth
timetable on a value that already held one kept both schedules. The single-value
branch of the same function has always done *x = BwTimetable{ts}, and the other
multi-token Set methods in this package build into a local and assign at the end.
The visible effect is through the rc API. The "main" options block registered in
fs.RegisterGlobalOptions is the live globalConfig, and options/set reshapes JSON
straight into it, so
rclone rc options/set --json '{"main": {"BwLimit": "Mon-10:00,1Mi"}}'
added to the running daemon's timetable rather than replacing it, and the older
slot kept winning: LimitAt for a Sunday returned the previous 10Mi. The same
applies to a _config override on a single call, since AddConfig shallow-copies
the global.
Building into a local also stops a failed parse from leaving the previous
timetable partly overwritten, which the existing error cases already expect.
Ceph RGW (and Linode Object Storage, which is Ceph-backed) can break
SigV4 when Accept-Encoding is included in the signature, especially
when a reverse proxy rewrites that header. GCS already sets this quirk;
apply the same default for Ceph and Linode as suggested in #8206.
Set the OCI source SSE-C request headers when using a customer key.
Server-side copies need these headers to decrypt the source object, in
addition to the existing headers that encrypt the destination.
The test asks InstallUpdate to install the latest beta into an
unwritable file and expects an error. When the binary under test
reports exactly the latest beta version (as make quicktest does right
after a beta is published from the same commit), InstallUpdate
correctly decides there is nothing to do and returns nil, and the test
then dereferences the nil error and panics.
Pin fs.Version to a fixed old value for the duration of the test so an
update is always attempted, and use require.Error so a missing error
fails the test instead of crashing it.
Decrypting the config with --daemon runs --password-command twice, which
means two authentications when the command needs one, such as a hardware
key touch for `pass show`.
SetConfigPassword saves the obscured key to the temp file named by
_RCLONE_CONFIG_KEY_FILE so the daemon process can pick it up, but the
process that wrote it then read and deleted that file itself before
daemonizing. The daemon started with the variable pointing at a file that
was already gone, found no key, and ran the password command again.
Skip acquiring a password when _RCLONE_CONFIG_KEY_FILE is set, as the
PassConfigKeyForDaemonization documentation already describes, and only
consume the key file in a process that has no key of its own. The parent
then leaves the key for the daemon, and the daemon uses it.
Fixes#7341
applyOptions consumes the "path" option into vol.Path rather than leaving
it in vol.Options, but restoreState rebuilt the options with only fs and
type. The explicit path was therefore dropped when the plugin restarted,
and since fsString is rebuilt from those options the volume was remounted
at the root of the remote instead of at its subpath.
Before this change a volume created with type + path lost its path
completely, and one created with remote + path silently fell back to the
path of the connection string. With a backend whose credentials are
scoped to the subpath the restored mount then failed every operation
rather than serving the wrong directory.
Feed the persisted path back like fs and type, so applyOptions applies
the same precedence on restore that it applies when the volume is
first created.
Fixes#9853
The Dropbox backend advertises CaseInsensitive: true, but the two
shared-mode lookup helpers compared names with an exact, case-sensitive
==, so a shared folder or received file named "Project" could not be
found when requested as "project". Use strings.EqualFold in both
findSharedFolder and findSharedFile to honour the advertised
case-insensitivity.
Fixes#9706
In shared_folders mode NewFs derived the shared folder name with
path.Dir(f.root), which returns the parent path rather than the first
path component. For a root like "SharedFolder/subdir/deeper" this yielded
"SharedFolder/subdir", which findSharedFolder cannot match, so NewFs
failed with ErrorDirNotFound. Use the first path component of the root,
as the shared_folders option documents, so deeply nested roots mount.
Fixes#9705
The flag applies to out of space errors while writing and while creating
files or directories, not only while writing. Describe those operations
without naming ENOSPC, which is a Unix error that Windows never reports, so
the help is accurate on every platform.
KickCleaner sets the out of space flag, kicks the cleaner and then waits for
that flag to clear. Only the cleaner clears it, and the cleaner returns
immediately when the cache poll interval is not positive, so when it is
disabled nothing ever reads the kick or clears the flag and the caller waits
forever.
It now returns straight away in that case, under the same condition the
cleaner itself uses to decide it is disabled. Callers already retry a bounded
number of times and then report the error, which is the right outcome when
nothing is going to free space.
IsErrNoSpace compared against syscall.ENOSPC. Go defines that constant on
Windows as a value in its application reserved range which no Windows API
returns, so the comparison could never be true there. A full disk on Windows
reports ERROR_DISK_FULL or ERROR_HANDLE_DISK_FULL instead.
Preallocation failures were still caught, because those return a separate
sentinel, but a disk that is already full fails at the directory creation or
at the open long before preallocation is reached. That is the case reported.
The errors are now held in a list which platform specific files add to in
their init, which is the shape retriable_errors already uses in this package,
and the comparison itself is unchanged. Windows appends the two codes that
lib/file already recognises when preallocation fails. Every other platform
keeps exactly the behaviour it had.
This also reaches the VFS cache, which uses the same helper and has no
preallocation path of its own, so its out of space handling has been inert
on Windows.
Google now requires an app homepage URL and privacy policy URL to be
set on the OAuth consent screen's "Branding" page before the "PUBLISH
APP" button becomes clickable, even for a personal single-user app.
The existing instructions jumped straight to publishing in step 9
without mentioning this, leaving the button greyed out with no
explanation of why.
Fixes#9854
Wang Chencheng