Objectives:
- Facilitate the generation of valid password hashes that can be used with
the `server user --user-password` CLI command.
- Encapsulate implementation details of password hashing in
the `user` package.
Adds a new `server user hash-password` CLI command to generate the
hash from a supplied password.
Modifies the `server user set/add --user-password-hash` CLI command
to accept the password hash generated using the `hash-password`
command.
Adds `GetNewProfile(ctx, rep, username)` helper to move implementation
details to the `user` package.
Includes CLI and unit tests.
Cleans up and removes unused functions.
Ensure repository disconnection at the end of the `server start` CLI command.
This was caught as a result of fixing the test below.
Fix `TestServerStartInsecure`:
Remove `--password=xxx` parameter, which causes a server start failure
due to incorrect repo password, and not for the case being checked,
which is the lack of the `--insecure` parameter.
Update test comments accordingly.
Use non-formatting logging functions for message without formatting.
For example, `log.Info("message")` instead of `log.Infof("message")`
Configure linter for printf-like functions
Code movement and simplification, no functional changes.
Objectives:
- Allow callers specifying the needed key (or hash) size, instead of
hard-coding it in the registered PBK derivers. Conceptually, the caller
needs to specify the key size, since that is a requirement of the
(encryption) algorithm being used in the caller. Now, the code changes
here do not result in any functional changes since the key size is
always 32 bytes.
- Remove a global definition for the default PB key deriver to use.
Instead, each of the 3 use case sets the default value.
Changes:
- `crypto.DeriveKeyFromPassword` now takes a key size.
- Adds new constants for the key sizes at the callers.
- Removes the global `crypto.MasterKeySize` const.
- Removes the global `crypto.DefaultKeyDerivationAlgorithm` const.
- Adds const for the default derivation algorithms for each use case.
- Adds a const for the salt length in the `internal/user` package, to ensure
the same salt length is used in both hash versions.
- Unexports various functions, variables and constants in the `internal/crypto`
& `internal/user` packages.
- Renames various constants for consistency.
- Removes unused functions and symbols.
- Renames files to be consistent and better reflect the structure of the code.
- Adds a couple of tests to ensure the const values are in sync and supported.
- Fixes a couple of typos
Followups to:
- #3725
- #3770
- #3779
- #3799
- #3816
The individual commits show the code transformations to simplify the
review of the changes.
Reverts to using the `PasswordHashVersion` in the user profile.
Adds a simple mechanism for translating between password hash
version and the corresponding password hashing algorithm (key
derivation algorithm).
* User profile add set cli changes
* Add additional test
* get salt length from key deriver
* Fixes for failing tests
* after shikhars review
* lint fixes
* Fix restoring objects with I prefix
set default of snapshot-time to 'latest' as noted in the help output
* Change test of restore to check it works without a time given
This is because --snapshot-time defaults to "latest" now.
* fixup for ticket #3624
* skip, now broken, test
* lint fixup. how did this get into the repository in the first place?!
* elide shadow copy tests until fixup. do not allow release
* fixup for shadow copy mode from @small
* Update cli/command_policy_set_os_snapshot_test.go
* restore old tests
* fixup typo
---------
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
* Implement volume shadow copy support on Windows
* Update go-vss version
* Fix unused variables
* Rename upload_actions*.go files
* Move vss settings to a separate policy section
* Handle existing shadow copy root
* Fix tests
* Fix lint issues
* Add cli policy test
* Add OS snapshot integration test
* Add GitHub Actions VSS test
* Fix "Incorrect function" error for root VSS snapshots
* Rename err to finalErr in createOSSnapshot
* Add OSSnapshotMode test
* Do not modify paths starting with \\?\ on Windows
* Allow warning messages in logfile tests
* Fix ignorefs not wrapping OS snapshot directory
* Retry VSS creation if another op was in progress
---------
Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
* refactor(test): allow signaling sub-process from testenv.CLIExeRunner
* test(cli): add test for handling SIGTERM
* feat(general): catch and process SIGTERM for termination
* refactor(cli): rename function cli.App.onTerminate
Renames function from onCtrlC to a more generic onTerminate
My backup had a fatal error. The end of the log looked like this:
```
Created snapshot with root k5ab05dd5a8aaf9da8a6a822abd0afabb and ID 04caa6e10f4e2866a74492a4162ea943 in 2m44s
WARN Ignored 943 error(s) while snapshotting root@tower:/.
ERROR Found 1 fatal error(s) while snapshotting root@tower:/.
```
Note that "WARN" is yellow and "ERROR" is red.
Since I got a fatal error, I wanted to check what the fatal error was.
Because "ERROR" in the lines above is red, I expected the fatal error in
the kopia log to also be red, but it was yellow like the non-fatal
errors. This was unexpected to me.
Also note that I have lots of "! Ignored error when processing" in the
kopia log because I also backup Docker containers, so right now it is
not easy to find the fatal error among the non-fatal errors.
* fix(repository): fixed handling of content.Info
Previously content.Info was an interface which was implemented by:
* index.InfoStruct
* index.indexEntryInfoV1
* index.indexEntryInfoV2
The last 2 implementations were relying on memory-mapped files
which in rare cases could be closed while Kopia was still processing
them leading to #2599.
This changes fixes the bug and strictly separates content.Info (which
is now always a struct) from the other two (which were renamed as
index.InfoReader and only used inside repo/content/...).
In addition to being safer, this _should_ reduce memory allocations.
* reduce the size of content.Info with proper alignment.
* pr feedback
* renamed index.InfoStruct to index.Info
* Improve RunMissed algorithm to work better with Cron and to give more predictable results for time-of-day rules
* Add a RunMissed test for multiple times-of-day
* add variable to improve code-readability
* Fix test after rebase
* feat(server): reduce server refreshes of the repository
Previously each source would refresh itself from the repository
very frequently to determine the upcoming snapshot time. This change
refactors source manager so it does not own the repository connection
on its own but instead delegates all policy reads through the server.
Also introduces a new server scheduler that is responsible for
centrally managing the snapshot schedule and triggering snapshots
when they are due.
* Update cli/command_server_start.go
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
* Update internal/server/server.go
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
* Update internal/server/server_maintenance.go
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
* pr feedback
---------
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
Added improved providervalidation logic which tests for read-after-write
property between connections. The new test was failing before the change
and is now passing for Google Drive, OneDrive and DropBox.
* feat(repository): apply retention policies server-side
This allows append-only snapshots where the client can never delete
arbitrary manifests and policies are maintained on the server.
The client only needs permissions to create snapshots in a given, which
automatically gives them permission to invoke the server-side method
for their own snapshots only.
* Update cli/command_acl_add.go
Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
* Update internal/server/api_manifest.go
Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
* Update internal/server/api_manifest.go
Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
* Update internal/server/grpc_session.go
Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
---------
Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
Previously some logs from a running server were only kept in memory
(including storage activity logs) which was confusing to many folks.
This changes the behavior so that logs are sent to their regular
(console/file) file locations in addition to the UI tasks.
Old behavior can be restored by adding `--no-persistent-logs` to
server.
