Remove `repositoryAccessMode.mustBeConnected`
It is always true.
Rename `repositoryAccessMode.disableMaintenance`
to `allowMaintenance`.
This explicitly conveys when maintenance is allowed
to run. It is related to accessing a repository in
'read-only' mode.
Allow the use of a client certificate when authenticating
to an Azure Blob storage provider.
Tests included.
Credit: @DeepikaDixit
Authored-by: Deepika Dixit <DeepikaDixit@users.noreply.github.com>
* use uint8 for clarity
* unexport writeContentAsyncAndVerify
* fix typo in test function name
* remove commented interface functions
* use atomic.Int32
* cleanups in socket server activation test
* leverage stdlib's maps and slices packages
replace uses of `golang.org/x/exp/maps`
* nit: leverage `maps.Values`
Extends error message with a generic, OS-independent hint about
what the source of the error may be and a potential solution.
Also, modifies error message to avoid duplicate messages.
- kopia/kopia#4449
Added functionality to calculate aggregate statistics when
comparing what's changed between snapshots using kopia diff
Statistics collected during snapshot diff computation includes:
- files added/removed/modified
- dirs added/removed/modified
- files/dirs with metadata changes but same underlying content (OID)
Testing approach:
Added a test for verifying stats collected when comparing two directories with the same objectID but metadata changes across snapshots (dir mode, dir mod time, dir owner, etc), expectation is all the appropriate dir stats fields are updated.
Added another test for verifying stats collected when comparing two directories with similar file contents but the metadata for the files have changed between snapshots but content remains unchanged. Expectation is all the relevant file level stats fields are updated.
Existing tests have been updated due to stats now being printed in addition to previous output.
* feat(general): various notifications improvements
* added API to test notification profiles
* added --http-header to webhook notification configuration
* refactored configuration to always apply defaults before persisting options in the repository
* added 'notification profile show --profile-name=X' command
* more tests
* more test coverage
* report notification code coverage
This was caused by the client using key derivation algorithm
from a config file (which did not have it when it was generated
using old version of Kopia).
Fixes#4254
* feat(cli): send error notifications and snapshot reports
Notifications will be sent to all configured notification profiles
according to their severity levels.
The following events will trigger notifications:
- Snapshot is created (CLI only, severity >= report)
- Server Maintenance error occurs (CLI, server and UI, severity >= error)
- Any other CLI error occurs (CLI only, severity >= error).
A flag `--no-error-notifications` can be used to disable error notifications.
* added template tests
* improved time formatting in templates
* plumb through notifytemplate.Options
* more testing for formatting options
* fixed default date format to RFC1123
* Add rough estimation mechanism
* Extract data size estimation scan to standalone function
* Introduce estimation type flag
* Change signature of EstimatedDataSize
* Extract estimation logic and test it
* Adjust err handling in GetVolumeSizeInfo
* Make GetVolumeSizeInfo reusable
* Fix type issue for windows platform
* Use adaptive estimation with threshold
* Configure compressor for k and x prefixed content
Adds metadata compression setting to policy
Add support to configure compressor for k and x prefixed content
Set zstd-fastest as the default compressor for metadata in the policy
Adds support to set and show metadata compression to kopia policy commands
Adds metadata compression config to dir writer
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Pass concatenate options with ConcatenateOptions struct
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move content compression handling to caller
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move handling manifests to manifest pkg
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Correct const in server_test
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Remove unnecessary whitespace
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Disable metadata compression for < V2 format
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
---------
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
Checks whether any of the `httpServer.Serve*()` calls returns
`ErrServerClosed`.
Handles `ErrServerClosed` inside the `startServerWithOptionalTLS` function
instead of propagating it up. This means that `startServerWithOptionalTLS`
returns a nil error when the HTTP server is closed, so the caller does not need
to check for `ErrServerClosed`
No functional changes otherwise.
Snapshot verify can now accept a list of one or more snapshot IDs (manifest IDs) to be used as the subject(s) of the snapshot verify operation.
Added a unit test that ensures only the requested snapshots are verified when their IDs are provided.
Now in addition to the existing --source, --file-id, and --directory-id flags, which work the same, the user can provide snapshot IDs as args:
kopia snapshot verify <snapID> <snapID> <snapID>
Followups to #3655
* wrap fs.Reader
* nit: remove unnecessary intermediate variable
* nit: rename local variable
* cleanup: move restore.Progress interface to cli pkg
* move cliRestoreProgress to a separate file
* refactor(general): replace switch with if/else for clarity
Removes a tautology for `err == nil`, which was guaranteed
to be true in the second case statement for the switch.
Replacing the switch statement with and if/else block is clearer.
* initialize restoreProgress in restore command
* fix: use error.Wrapf with format string and args
Simplify SetCounters signature:
Pass arguments in a `restore.Stats` struct.
`SetCounters(s restore.Stats)`
Simplifies call sites and implementation.
In this case it makes sense to pass all the values
using the restore.Stats struct as it simplifies
the calls.
However, this pattern should be avoided in general
as it essentially makes all the arguments "optional".
This makes it easy to miss setting a value and simply
passing 0 (the default value), thus it becomes error
prone.
In this particular case, the struct is being passed
through verbatim, thus eliminating the risk of
missing a value, at least in the current state of
the code.
Generalize a couple of functions in the units package using generics.
This allows removing duplicate code and simplifying callers by removing unnecessary integer conversions.
Additional cleanups:
- make "/s" part of the Printf format string ;
- simplify setSizeMBParameter;
- generalize cli.maybeHumanReadable*` helpers;
- remove unneeded receiver in commandRepositorySetParameters helpers.
Cleanups:
- use non-format variants of Log/Print with no additional args;
- fold in Fprintf call with no args into the following one;
- add missing arg placeholder in format strings;
- use require.Positive instead of Greater(..., 0);
- rename function to fillWithZeros to avoid collision with builtin clear;
- define type for context key to avoid collisions.
Objectives:
- Facilitate the generation of valid password hashes that can be used with
the `server user --user-password` CLI command.
- Encapsulate implementation details of password hashing in
the `user` package.
Adds a new `server user hash-password` CLI command to generate the
hash from a supplied password.
Modifies the `server user set/add --user-password-hash` CLI command
to accept the password hash generated using the `hash-password`
command.
Adds `GetNewProfile(ctx, rep, username)` helper to move implementation
details to the `user` package.
Includes CLI and unit tests.
Cleans up and removes unused functions.
Ensure repository disconnection at the end of the `server start` CLI command.
This was caught as a result of fixing the test below.
Fix `TestServerStartInsecure`:
Remove `--password=xxx` parameter, which causes a server start failure
due to incorrect repo password, and not for the case being checked,
which is the lack of the `--insecure` parameter.
Update test comments accordingly.
Use non-formatting logging functions for message without formatting.
For example, `log.Info("message")` instead of `log.Infof("message")`
Configure linter for printf-like functions
