_logs can now be set to an object, e.g.
{"level": "INFO", "unattributed": false}
Setting unattributed to false leaves out the logs which aren't
attributed to a job, so a client can have just the logs of its own
job. It defaults to true which is what _logs did before.
Every rc call and its reply is logged at DEBUG level, so a client
reading the logs over the rc with core/events or core/log sees the
logs of its own polling, which can drown out everything else.
Setting --rc-log-calls=false stops rclone logging the calls. It can
be changed while rclone is running with options/set.
This adds core/events which streams events over a single HTTP
connection as newline delimited JSON, one object per line, each with
a type. Log entries are sent as they are made with type "log" in the
same format as core/log returns, so they can be attributed to the rc
job which made them.
This doesn't need the log buffer to be enabled as the events aren't
stored, but if it is enabled then the since parameter sends the
entries from the buffer first so a client which reconnects doesn't
miss any.
The sequence numbers and the JSON for the log entries are now made
once for the log buffer and the subscribers to share.
Log entries which rclone can attribute to the rc job which made them
now have jobid and group entries in the JSON log. core/log can filter
on these with the new group and jobid parameters.
_logs and job/status now return the logs of the job itself plus the
logs made while it was running which aren't attributed to any job,
leaving out the logs of other jobs running at the same time. Note
that calls sharing a stats group with _group are attributed
separately as they are different jobs.
If _logs is set to true or a log level on an rc call then the logs
made while the call was running are returned as _logs in the result,
or in the error response if the call failed. For _async calls the logs
so far are returned as logs in job/status.
This needs the log buffer enabling with --log-buffer-size.
Logs aren't currently attributed to the call which made them so this
returns everything logged while the call was running. Because of this
_logs is refused on an rc server without authentication so the logs
can't be read through the calls which don't need authentication.
This adds the --log-buffer-size flag which keeps the most recent logs
in memory in JSON format, dropping the oldest when it is full.
These can then be read with the new rc call core/log which returns
the log entries from a given sequence number so it can be polled to
follow the log.
The log buffer is disabled by default and can be enabled at runtime
with options/set.
Use OCI BatchDeleteObjects to remove up to 1,000 objects per request when
purging. Include directory markers so purging a bucket leaves it empty for
deletion.
Use OCI RenameObject for moves within a bucket, avoiding a server-side copy
followed by source deletion. Cross-bucket moves continue to fall back to copy
and delete.
The link pointed at a bare relative path (--check-filename) instead of
the in-page anchor for the ### --check-filename heading further down the
page, so it 404s on the rendered docs site. Point it at #check-filename
(Hugo strips leading dashes when generating header anchors).
---------
Co-authored-by: no-hup <19599684+no-hup@users.noreply.github.com>
Ceph RGW (and Linode Object Storage, which is Ceph-backed) can break
SigV4 when Accept-Encoding is included in the signature, especially
when a reverse proxy rewrites that header. GCS already sets this quirk;
apply the same default for Ceph and Linode as suggested in #8206.
Google now requires an app homepage URL and privacy policy URL to be
set on the OAuth consent screen's "Branding" page before the "PUBLISH
APP" button becomes clickable, even for a personal single-user app.
The existing instructions jumped straight to publishing in step 9
without mentioning this, leaving the button greyed out with no
explanation of why.
Fixes#9854
Directory names which look like they have a --b2-versions version
string are now encrypted in full, so directories created by older
rclone (which left the version string in plain text) no longer
decrypt and vanished silently from listings.
DecryptDirName now falls back to the old form for such names so the
directory is listed, and logs the name it needs to be renamed to on
the underlying remote to make it accessible again. Document this in
the crypt docs.
Several documentation links pointed at anchors or paths that no longer
resolve, and the S3 directory buckets section named the config option
and flag in the plural, which does not match the backend.
Co-authored-by: shaurya <19599684+no-hup@users.noreply.github.com>
Co-authored-by: no-hup <shauryaj.finance@gmail.com>
Nextcloud only stores a checksum which is supplied in the OC-Checksum
header of an upload, and discards it again when the modification time
is set with PROPPATCH. Re-sending the checksum in the PROPPATCH (as is
done for ownCloud) is rejected by Nextcloud with 403 Forbidden which
made the whole PROPPATCH fail, so SetModTime returned an error on any
object which had a hash. Uploads from sources without hashes, eg
streamed uploads with `rclone rcat`, were stored with no hash at all.
Use the Nextcloud PATCH extension with the X-Recalculate-Hash header
to have the server calculate and store the SHA1 of an object after a
streamed upload and after setting the modification time. This gives
a server side hash of the stored data which also lets rclone verify
streamed uploads.
Nick Craig-Wood