Previously, empty master keys were passed to the underlying
cryptographic primitives (HKDF, AEAD, etc.).
While this worked because the authentication mechanisms returned an
error, it's best to avoid passing empty master keys to these primitives
in the first place. This refactor avoids passing empty master keys and
enforces this via an assertion in the key derivation function.
Code movement and simplification, no functional changes.
Objectives:
- Allow callers specifying the needed key (or hash) size, instead of
hard-coding it in the registered PBK derivers. Conceptually, the caller
needs to specify the key size, since that is a requirement of the
(encryption) algorithm being used in the caller. Now, the code changes
here do not result in any functional changes since the key size is
always 32 bytes.
- Remove a global definition for the default PB key deriver to use.
Instead, each of the 3 use case sets the default value.
Changes:
- `crypto.DeriveKeyFromPassword` now takes a key size.
- Adds new constants for the key sizes at the callers.
- Removes the global `crypto.MasterKeySize` const.
- Removes the global `crypto.DefaultKeyDerivationAlgorithm` const.
- Adds const for the default derivation algorithms for each use case.
- Adds a const for the salt length in the `internal/user` package, to ensure
the same salt length is used in both hash versions.
- Unexports various functions, variables and constants in the `internal/crypto`
& `internal/user` packages.
- Renames various constants for consistency.
- Removes unused functions and symbols.
- Renames files to be consistent and better reflect the structure of the code.
- Adds a couple of tests to ensure the const values are in sync and supported.
- Fixes a couple of typos
Followups to:
- #3725
- #3770
- #3779
- #3799
- #3816
The individual commits show the code transformations to simplify the
review of the changes.
Reverts to using the `PasswordHashVersion` in the user profile.
Adds a simple mechanism for translating between password hash
version and the corresponding password hashing algorithm (key
derivation algorithm).
* User profile add set cli changes
* Add additional test
* get salt length from key deriver
* Fixes for failing tests
* after shikhars review
* lint fixes
* Add multi-language support
* Adding a guidline to outline the localisation process.
* Moved doc to separate pr
---------
Co-authored-by: lupusA <lupuapps@gmail.com>
Avoid minio allocations by implementing `ReadAt` function.
This change causes Minio libraries to reduce memory allocations.
Specifically, buffer allocations for S3 upload.
Add tests for new functionality.
---------
Co-authored-by: Shikhar Mall <shikhar.mall@veeam.com>
Co-authored-by: Nick <nick@kasten.io>
Change default 'allowWritesOnIndexLoad' to false when env var is unset
Add TestNoEpochAdvanceOnIndexRead
Ref:
- Followup to #3645
- Avoid index (epoch) cleanup and compaction during index reads #3638
- Make "read" commands/operations really read-only. #3639
Add:
- epoch.Manager.MaybeCompactSingleEpoch
- getCompactedEpochRange helper
- oldestUncompactedEpoch helper
- TestOldestUncompactedEpoch
- Tests for MaybeCompactSingleEpoch
Ref:
- Subset and dependency of #3651
- Depends on #3735
- Avoid index (epoch) cleanup and compaction during index reads #3638
- Make "read" commands/operations really read-only. #3639
Refactoring for the original implementation with intRange and
getKeyRange from closed-open ranges [lo, hi) to closed ranges: [lo, hi].
The primary motivation is for consistency with the implementation
of epoch.RangeMetadata in the same package, and thus avoid
confusion and reduce cognitive load.
Changes:
- adds a getContiguousKeyRange wrapper that checks for contiguity.
- getKeyRange simply returns a range with minimum and maximum
values for the keys in the map.
- changes the range implementation from closed-open ranges [lo, hi)
to closed ranges: [lo, hi] where both lo and hi are included in the range.
- Additional unit tests are included.
- renames intRange to closedIntRange to reflect new functionality.
Ref:
- Follow up refactor(general): add epoch.getKeyRange helper #3721
- Needed for refactor(general): add epoch.Manager.MaybeCompactSingleEpoch #3728
- Avoid index (epoch) cleanup and compaction during index reads #3638
* feat(repository): added metrics timeseries computation
This code is not used anywhere yet, will be used for reporting over
repository metrics.
* fixed checklocks
* Update internal/metrics/metrics_timeseries.go
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
---------
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
Conditionally disables epoch index maintenance operations when loading indexes.
This prevents (potentially expensive) cleanup write operations on the index
read path.
The behavior is controlled via the `epoch.Manager.allowCleanupWritesOnIndexLoad`
field, which can be temporarily overridden via an environment variable.
This override mechanism will be removed in the near future.
Refs:
- #3174
- #3224
- #3225
- #3638
- #3639
Refactor: move index compaction and cleanup out of refreshAttemptLocked
Introduces an `allowWritesOnLoadHelper` to check whether or not writes
can be performed when loading the indexes. Currently this is only
a function of whether the storage is in read-only mode. In the near
future, an explicit flag will be added to control this behavior.
Fix epoch manager: avoid single-epoch compaction when writes are disallowed.
Functional change: prevents compacting single epochs when writes are
disallowed, that is when using read-only storage. Currently, the epoch
manager will attempt to perform single-epoch compactions for all
eligible epochs, even on read-only storage.
Ref:
- #3224
- #3225
- #3638
- #3639
* Implement volume shadow copy support on Windows
* Update go-vss version
* Fix unused variables
* Rename upload_actions*.go files
* Move vss settings to a separate policy section
* Handle existing shadow copy root
* Fix tests
* Fix lint issues
* Add cli policy test
* Add OS snapshot integration test
* Add GitHub Actions VSS test
* Fix "Incorrect function" error for root VSS snapshots
* Rename err to finalErr in createOSSnapshot
* Add OSSnapshotMode test
* Do not modify paths starting with \\?\ on Windows
* Allow warning messages in logfile tests
* Fix ignorefs not wrapping OS snapshot directory
* Retry VSS creation if another op was in progress
---------
Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
