* [Robustness] Fix for kopia runner and custom work dir
Apply fix similar to #293 for the robustness kopia runner.
Add control for runner working directory.
Fix fswalker to ignore hostname to allow reporting
on walks done across different hosts. Also prevent
Before and After walk data from printing to reduce log size.
* cli: small tweaks to kopia server mode
* print SHA256 certficate thumbprint for auto-generated certs.
* client will accept both upper- and lowercase thumbprint values
* site: updated documentation for v0.6.0 release
Co-authored-by: Julio López <julio+gh@kasten.io>
This creates a marker file named `.kopia-cache` in the directory
that is the root of cache. When the uploader finds this file, it will
treat the entire directory as if it were empty.
This allows excluding directory caches from entire home and root
directories.
* blob: added DisplayName() method to blob.Storage
* cli: added 'kopia repo sync-to <provider>' which replicates BLOBs
Usage demo: https://asciinema.org/a/352299Fixes#509
* implemented suggestion by Ciantic to fail sync if the destination repository is not compatible with the source
* cli: added 'kopia repo sync --must-exist'
This ensures that target repository is not empty, otherwise syncing to
an accidentally unmounted filesystem directory might copy everything
again.
* cli: fixed 'kopia policy rm' deleting global when passed policy ID
* policy: additional unit test coverage for policy manager
* fixed path parsing logic to avoid the use filepath package which is platform-dependent, added more tests
Proposed fix for #516: Do not attempt to call `os.Chown` on Windows. The
command is unsupported on windows and will always return
`syscall.EWINDOWS`.
Handling for user ID and group ID on Windows already hardcodes
both values to zero. If a snapshot is restored with non-zero UID/GID
values (for instance if the snapshot was created on a different OS),
filesystem entries will be restored without changing UID/GID, resulting
in a state similar to how they would look if the snapshot was
taken on Windows in the first place.
Fix parsing of pack blob ID by using a specific regex
instead of a strings.Contains. This prevents the test from
misidentifying other blob IDs as pack blobs, such as
`kopia.maintenance`.
* goreleaser: upgraded to v1.140.1
* site: upgrade to latest Hugo, fixed deprecation warnings
* app: upgraded NPM dependencies to latest version to address some low-severity vulnerabilities
* fixed a number of cases where misaligned data was causing panics on armv7 (but not armv8)
* travis: enable arm64
* test: reduce compressed data sizes when running on arm
* arm: wait longer for snapshots
KOPIA_VERSION will now always be v-prefixed and we will strip the
prefix before embedding it in KopiaUI manifest.
Also upgraded Node and app NPM dependencies to latest versions.
* restore: support for zip, tar and tar.gz restore outputs
Moved restore functionality to its own package.
* Fix enum values in the 'mode' flag
Co-authored-by: Julio López <julio+gh@kasten.io>
Add two tests:
- TestManySmallFiles: writes 100k files size 4k to a directory. Snapshots the data tree, restores and validates data.
- TestModifyWorkload: Loops over a simple randomized workload. Performs a series of random file writes to some random sub-directories, then takes a snapshot of the data tree. All snapshots taken during this test are restore-verified at the end.
A global test engine is instantiated in main_test.go, to be used in the robustness test suite across tests (saves time loading/saving metadata once per run instead of per test).
* Add test engine to manage snapshot verification testing
Test engine manages the test and metadata repositories, snapshot
checker, metadata storage persistence, and file writer. It is
the high level helper that will be invoked in the snapshot
verification testing suite.
- modify data directory file structure
- issue snapshot/restore/delete to the data directory
- accumulate metadata over the course of the test suite
- flush accumulated metadata to the metadata repository
- load historical metadata from the repository on initialization
- perform automatic data integrity verification on snap restore
This change corresponds to the robustness execution engine component from the design documentation.
* content: ensure that cleanup blobs have unique contents to prevent situation where they keep getting rewritten and thus never deleted
* cli: added '--decrypt' option to 'kopia blob show'
Added test that verifies that when client performs Flush (which happens
at the end of each snapshot and when repository is closed), the
server writes new blobs to the storage.
Fixes#464
- run maintenance even if the command is about to return an error
(otherwise if folks have persistent error causing snapshots to fail
they will never run maintenance)
- disable progress output after snapshotting so that
'kopia snapshot --all' output is clean
Add sftp and webdav as repositories to "Getting started" documentation page, "Setting Up Repository" chapter.
Add repositories list and usage examples to doc.
instead moved to run as part of maintenance ('kopia maintenance run')
added 'kopia maintenance run --force' flag which runs maintenance even
if not owned
