- use snapshot end time as tie breaker for equal start time
- pass prototype snapshot manifest in uploader
- remove -1 ns hack for checkpoints and fix corresponding test
- test snapshot.SortByTime
- use `slices.Clone`
- remove stale `.gometalinter.json`
- unexport `maintenance.dropDeletedContents`
- rename `fetchIndexBlob`
- use `require` in `TestTimeFuncWiring`
- Add read stats to snapshot verifier output
- Add periodic JSON progress output.
- Refactor the use of directory summary.
- Use stats mutex for all stats.
- Add processedBytes to the snapshot verify output
- Output more frequently, when bytes processed changes
* remove unnecessary type argument
* modernize with max
* unexport getPartial and update comment
* unexport getFull
* verifyNotCached helper
* use require helpers
* leverage verify[Not]Cached
* use windowsOSName const
* fix comment wrapping
* require in stat_test
* use 512 as the write size and log allocated size
* rename const to expectedMinAllocSize
* write a single byte to test file
* add TestGetBlockSizeFromCurrentFS
* require Positive
* log before invariant check
- enable `forcetypeassert` linter in non-test files
- add `//nolint` annotations
- add `testutil.EnsureType` helper for type assertions
- enable `forcetypeassert` linter in test files
- use 'require/assert'
- refactor TestUploadMetadataCompression as a table test
- allow tests to run in parallel
- use t.Cleanup and add a missing cleanup
- use maps.Clone
Cleanups in snapshot GC:
- add findUnreferencedAndRepairRereferenced
- moves logging from snapshotgc.Run to findUnreferencedAndRepairRereferenced
- removes returning collected stats, simplifies signature
- simplifies caller (snapshotgc.Run)
- removes now unused snapshotgc.Stats
Changes in behavior:
- logs before flushing to get info even if flush fails
- new: logs undeleted content stats
* fix(snapshots): Remove checkpoints after a complete snapshot
... by setting start time of checkpoints one nanosecond earlier than
that of the snapshot.
* test(snapshots): Test for leftover checkpoints
* fix linter issues
* removed stray curly brace
---------
Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
Avoid starting a maintenance task on the server when
the repository connection (configuration) is read-only.
Also:
* Check for read-only repo before running maintenance.
* Move direct repo check to startMaintenanceManager.
* Rename function as maybeStartMaintenanceManager.
The name reflects that the function may not start a
maintenance task manager.
* Add connect options to repotesting.
* Add test for maintenance on read-only repo.
- Fixes: #4373
* Add rough estimation mechanism
* Extract data size estimation scan to standalone function
* Introduce estimation type flag
* Change signature of EstimatedDataSize
* Extract estimation logic and test it
* Adjust err handling in GetVolumeSizeInfo
* Make GetVolumeSizeInfo reusable
* Fix type issue for windows platform
* Use adaptive estimation with threshold
* Configure compressor for k and x prefixed content
Adds metadata compression setting to policy
Add support to configure compressor for k and x prefixed content
Set zstd-fastest as the default compressor for metadata in the policy
Adds support to set and show metadata compression to kopia policy commands
Adds metadata compression config to dir writer
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Pass concatenate options with ConcatenateOptions struct
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move content compression handling to caller
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Move handling manifests to manifest pkg
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Correct const in server_test
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Remove unnecessary whitespace
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
* Disable metadata compression for < V2 format
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
---------
Signed-off-by: Prasad Ghangal <prasad.ganghal@veeam.com>
Followups to #3655
* wrap fs.Reader
* nit: remove unnecessary intermediate variable
* nit: rename local variable
* cleanup: move restore.Progress interface to cli pkg
* move cliRestoreProgress to a separate file
* refactor(general): replace switch with if/else for clarity
Removes a tautology for `err == nil`, which was guaranteed
to be true in the second case statement for the switch.
Replacing the switch statement with and if/else block is clearer.
* initialize restoreProgress in restore command
* fix: use error.Wrapf with format string and args
Simplify SetCounters signature:
Pass arguments in a `restore.Stats` struct.
`SetCounters(s restore.Stats)`
Simplifies call sites and implementation.
In this case it makes sense to pass all the values
using the restore.Stats struct as it simplifies
the calls.
However, this pattern should be avoided in general
as it essentially makes all the arguments "optional".
This makes it easy to miss setting a value and simply
passing 0 (the default value), thus it becomes error
prone.
In this particular case, the struct is being passed
through verbatim, thus eliminating the risk of
missing a value, at least in the current state of
the code.
Cleanups:
- use non-format variants of Log/Print with no additional args;
- fold in Fprintf call with no args into the following one;
- add missing arg placeholder in format strings;
- use require.Positive instead of Greater(..., 0);
- rename function to fillWithZeros to avoid collision with builtin clear;
- define type for context key to avoid collisions.
Use non-formatting logging functions for message without formatting.
For example, `log.Info("message")` instead of `log.Infof("message")`
Configure linter for printf-like functions
* fixup for ticket #3624
* skip, now broken, test
* lint fixup. how did this get into the repository in the first place?!
* elide shadow copy tests until fixup. do not allow release
* fixup for shadow copy mode from @small
* Update cli/command_policy_set_os_snapshot_test.go
* restore old tests
* fixup typo
---------
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
* Implement volume shadow copy support on Windows
* Update go-vss version
* Fix unused variables
* Rename upload_actions*.go files
* Move vss settings to a separate policy section
* Handle existing shadow copy root
* Fix tests
* Fix lint issues
* Add cli policy test
* Add OS snapshot integration test
* Add GitHub Actions VSS test
* Fix "Incorrect function" error for root VSS snapshots
* Rename err to finalErr in createOSSnapshot
* Add OSSnapshotMode test
* Do not modify paths starting with \\?\ on Windows
* Allow warning messages in logfile tests
* Fix ignorefs not wrapping OS snapshot directory
* Retry VSS creation if another op was in progress
---------
Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
Rename faketime.AutoAdvance parameter to start for clarity
Clarify faketime.ClockTimeWithOffset.Advance documentation
Refactor `faketime.TimeAdvance`:
Add `faketime.NewAutoAdvance` to create auto-advancing
`TimeAdvance` clocks.
Refactor faketime.NewTimeAdvance to return non-auto-advancing clocks,
which can still be explicitly advanced.
Removes the now unused `autoDelta` parameter, since it is always 0.
Rename function to `sm.setupCachesAndIndexManagers`
It matches the current implementation better.
Unexport `indexblob.ManagerV1.epochMgr`
Rename function for clarity to `advanceEpochMarker`
Cleanup logs after cleaning up epoch manager.
This is consistent with quick maintenance as well.
Make 'def' a test-local variable instead of a package-level
variable, it is not used outside this test.
Cleanup epoch test newTestEnv()
Misc:
* Reword comment and fix typo
* Fix log message
* `log.Debug` for non-formatted output
* Improve RunMissed algorithm to work better with Cron and to give more predictable results for time-of-day rules
* Add a RunMissed test for multiple times-of-day
* add variable to improve code-readability
* Fix test after rebase
