- Add read stats to snapshot verifier output
- Add periodic JSON progress output.
- Refactor the use of directory summary.
- Use stats mutex for all stats.
- Add processedBytes to the snapshot verify output
- Output more frequently, when bytes processed changes
The `dirRelativePath` variable is actually the path to the file being
checked, but was treated as if it was the path to the parent directory,
causing the filename to be duplicated in log messages.
- enable `forcetypeassert` linter in non-test files
- add `//nolint` annotations
- add `testutil.EnsureType` helper for type assertions
- enable `forcetypeassert` linter in test files
- nit: rename var to packCountByPrefix
- leverage impossible package
- use maps.Clone
- unexport indirectObjectID
- unexport compressed
- rename function to flushBufferLocked
- add checklocks annotations to functions that must be called under w.mu
Adds a `--stats-only` flag to the `diff` command to reduce
output verbosity.
When `kopia diff` is invoked with the `--stats-only` flag it
only outputs the aggregate statistics of changes.
Remove `repositoryAccessMode.mustBeConnected`
It is always true.
Rename `repositoryAccessMode.disableMaintenance`
to `allowMaintenance`.
This explicitly conveys when maintenance is allowed
to run. It is related to accessing a repository in
'read-only' mode.
Allow the use of a client certificate when authenticating
to an Azure Blob storage provider.
Tests included.
Credit: @DeepikaDixit
Authored-by: Deepika Dixit <DeepikaDixit@users.noreply.github.com>
* use uint8 for clarity
* unexport writeContentAsyncAndVerify
* fix typo in test function name
* remove commented interface functions
* use atomic.Int32
* cleanups in socket server activation test
* leverage stdlib's maps and slices packages
replace uses of `golang.org/x/exp/maps`
* nit: leverage `maps.Values`
Extends error message with a generic, OS-independent hint about
what the source of the error may be and a potential solution.
Also, modifies error message to avoid duplicate messages.
- kopia/kopia#4449
Added functionality to calculate aggregate statistics when
comparing what's changed between snapshots using kopia diff
Statistics collected during snapshot diff computation includes:
- files added/removed/modified
- dirs added/removed/modified
- files/dirs with metadata changes but same underlying content (OID)
Testing approach:
Added a test for verifying stats collected when comparing two directories with the same objectID but metadata changes across snapshots (dir mode, dir mod time, dir owner, etc), expectation is all the appropriate dir stats fields are updated.
Added another test for verifying stats collected when comparing two directories with similar file contents but the metadata for the files have changed between snapshots but content remains unchanged. Expectation is all the relevant file level stats fields are updated.
Existing tests have been updated due to stats now being printed in addition to previous output.
* feat(general): various notifications improvements
* added API to test notification profiles
* added --http-header to webhook notification configuration
* refactored configuration to always apply defaults before persisting options in the repository
* added 'notification profile show --profile-name=X' command
* more tests
* more test coverage
* report notification code coverage
This was caused by the client using key derivation algorithm
from a config file (which did not have it when it was generated
using old version of Kopia).
Fixes#4254
* feat(cli): send error notifications and snapshot reports
Notifications will be sent to all configured notification profiles
according to their severity levels.
The following events will trigger notifications:
- Snapshot is created (CLI only, severity >= report)
- Server Maintenance error occurs (CLI, server and UI, severity >= error)
- Any other CLI error occurs (CLI only, severity >= error).
A flag `--no-error-notifications` can be used to disable error notifications.
* added template tests
* improved time formatting in templates
* plumb through notifytemplate.Options
* more testing for formatting options
* fixed default date format to RFC1123
* Add rough estimation mechanism
* Extract data size estimation scan to standalone function
* Introduce estimation type flag
* Change signature of EstimatedDataSize
* Extract estimation logic and test it
* Adjust err handling in GetVolumeSizeInfo
* Make GetVolumeSizeInfo reusable
* Fix type issue for windows platform
* Use adaptive estimation with threshold
* Configure compressor for k and x prefixed content
Adds metadata compression setting to policy
Add support to configure compressor for k and x prefixed content
Set zstd-fastest as the default compressor for metadata in the policy
Adds support to set and show metadata compression to kopia policy commands
Adds metadata compression config to dir writer
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Pass concatenate options with ConcatenateOptions struct
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move content compression handling to caller
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move handling manifests to manifest pkg
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Correct const in server_test
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Remove unnecessary whitespace
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Disable metadata compression for < V2 format
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
---------
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
Checks whether any of the `httpServer.Serve*()` calls returns
`ErrServerClosed`.
Handles `ErrServerClosed` inside the `startServerWithOptionalTLS` function
instead of propagating it up. This means that `startServerWithOptionalTLS`
returns a nil error when the HTTP server is closed, so the caller does not need
to check for `ErrServerClosed`
No functional changes otherwise.
