Adds a `--stats-only` flag to the `diff` command to reduce
output verbosity.
When `kopia diff` is invoked with the `--stats-only` flag it
only outputs the aggregate statistics of changes.
Added functionality to calculate aggregate statistics when
comparing what's changed between snapshots using kopia diff
Statistics collected during snapshot diff computation includes:
- files added/removed/modified
- dirs added/removed/modified
- files/dirs with metadata changes but same underlying content (OID)
Testing approach:
Added a test for verifying stats collected when comparing two directories with the same objectID but metadata changes across snapshots (dir mode, dir mod time, dir owner, etc), expectation is all the appropriate dir stats fields are updated.
Added another test for verifying stats collected when comparing two directories with similar file contents but the metadata for the files have changed between snapshots but content remains unchanged. Expectation is all the relevant file level stats fields are updated.
Existing tests have been updated due to stats now being printed in addition to previous output.
* feat(infra): improved support for in-process testing
* support for killing of a running server using simulated Ctrl-C
* support for overriding os.Stdin
* migrated many tests from the exe runner to in-process runner
* added required indirection when defining Envar() so we can later override it in tests
* refactored CLI runners by moving environment overrides to CLITestEnv
* cli: fixed remaining testability indirections for output and logging
* cli: added cli.RunSubcommand() which is used in testing to execute a subcommand in the same process
* tests: refactored most e2e tests to invoke kopia subcommands in-process
* Makefile: enable code coverage for cli/ and internal/
* testing: pass 'testing' tag to unit tests which uses much faster (insecure) password hashing scheme
* Makefile: push coverage from PRs again
* tests: disable buffer management to reduce memory usage on ARM
* cli: fixed misaligned atomic field on ARMHF
also temporarily fixed statup-time benign race condition when setting
default on the timeZone variable, which is the last global variable.
cli: major refactoring of how CLI commands are registered
The goal is to eliminate flags as global variables to allow for better
testing. Each command and subcommand and most sets of flags are now
their own struct with 'setup()' methods that attached the flags or
subcommand to the provided parent.
This change is 94.3% mechanical, but is fully organic and hand-made.
* introduced cli.appServices interface which provides the environment in which commands run
* remove auto-maintenance global flag
* removed globals in memory_tracking.go
* removed globals from cli_progress.go
* removed globals from the update_check.go
* moved configPath into TheApp
* removed remaining globals from config.go
* refactored logfile to get rid of global variables
* removed 'app' global variable
* linter fixes
* fixed password_*.go build
* fixed BSD build
- `repo.Repository` is now read-only and only has methods that can be supported over kopia server
- `repo.RepositoryWriter` has read-write methods that can be supported over kopia server
- `repo.DirectRepository` is read-only and contains all methods of `repo.Repository` plus some low-level methods for data inspection
- `repo.DirectRepositoryWriter` contains write methods for `repo.DirectRepository`
- `repo.Reader` removed and merged with `repo.Repository`
- `repo.Writer` became `repo.RepositoryWriter`
- `*repo.DirectRepository` struct became `repo.DirectRepository`
interface
Getting `{Direct}RepositoryWriter` requires using `NewWriter()` or `NewDirectWriter()` on a read-only repository and multiple simultaneous writers are supported at the same time, each writing to their own indexes and pack blobs.
`repo.Open` returns `repo.Repository` (which is also `repo.RepositoryWriter`).
* content: removed implicit flush on content manager close
* repo: added tests for WriteSession() and implicit flush behavior
* invalidate manifest manager after write session
* cli: disable maintenance in 'kopia server start'
Server will close the repository before completing.
* repo: unconditionally close RepositoryWriter in {Direct,}WriteSession
* repo: added panic in case somebody tries to create RepositoryWriter after closing repository
- used atomic to manage SharedManager.closed
* removed stale example
* linter: fixed spurious failures
Co-authored-by: Julio López <julio+gh@kasten.io>
* linter: upgraded to 1.33, disabled some linters
* lint: fixed 'errorlint' errors
This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.
Verified that there are no exceptions for errorlint linter which
guarantees that.
* lint: fixed or suppressed wrapcheck errors
* lint: nolintlint and misc cleanups
Co-authored-by: Julio López <julio+gh@kasten.io>
* restore: improved user experience
* 'snapshot restore' is now the same as 'restore' and both will
support restoring by manifest ID, root ID or root ID + subdirectory
* added support for restoring individual files
* implemented PR feedback and refactored object ID parsing
Moving helpers inside the snapshot/ package helped clean up the code
a lot.
* This is 99% mechanical:
Extracted repo.Repository interface that only exposes high-level object and manifest management methods, but not blob nor content management.
Renamed old *repo.Repository to *repo.DirectRepository
Reviewed codebase to only depend on repo.Repository as much as possible, but added way for low-level CLI commands to use DirectRepository.
* PR fixes
- Use it to compare the entry attributes for all entry types.
Allows comparing differences in file attributes among the
contents of two directories. This is useful for verifying
restored contents in end-to-end tests.
- Print message about modified entries only when both entries
are files and they are being compared.
