- upgrade to golangci-lint 2.6.1
- updates for gosec
- updates for govet
- updates for perfsprint
- updates modernize
Leaves out modernize:omitempty due to conflicts with tests
Add return values in long helper function. This Improves clarity
and avoids 'naked' return and removes `//nolint:nakedret` annotation.
Explicit return values for GetShardedPathAndFilePath
- nit: re-group struct fields
- nit: use consts
- nit: remove unnecessary fmt.Errorf(...) usage
- nit: avoid unnecessarily calling defaultActionControls when there is already a value
- robustness: increase readability in actions map declaration
- Prefer named functions over closures for the actions.
- nit fix typo
- nit: simplify robustness Log.StringThisRun
Iterates only over the tail of the slice, which avoids iterating over the entire slice
* fixed new gocritic violations
* fixed new 'contextcheck' violations
* fixed 'gosec' warnings
* suppressed ireturn and varnamelen linters
* fixed tenv violations, enabled building robustness tests on arm64
* fixed remaining linux failures
* makefile: fixed 'lint-all' target when running on arm64
* linter: increase deadline
* disable nilnil linter - to be enabled in separate PR
The dual time measurement is described in
https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md
The fix is to discard hidden monotonic time component of time.Time
by converting to unix time and back.
Reviewed usage of clock.Now() and replaced with timetrack.StartTimer()
when measuring time.
The problem in #1402 was that passage of time was measured using
the monotonic time and not wall clock time. When the computer goes
to sleep, monotonic time is still monotonic while wall clock time makes
a leap when the computer wakes up. This is the behavior that
epoch manager (and most other compontents in Kopia) rely upon.
Fixes#1402
Co-authored-by: Julio Lopez <julio+gh@kasten.io>
* refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* chore: remove //nolint:gosec for os.ReadFile
At the time of this commit, the G304 rule of gosec does not include the
`os.ReadFile` function. We remove `//nolint:gosec` temporarily until
https://github.com/securego/gosec/pull/706 is merged.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Detailed changes:
* Implement Kopia API client
* Implement kopia client
* Implement new persister
* Separate storeLoad and Delete concurrency tests
* Update Store interface to return error for Delete
* Return errors from os.RemoveAll
* Robustness test updates
* Push robustness metadata without writing to fs
* Fix testenv.AssertNoError references
* Update tests to use new kopia persister
* Minor updates to multiclient test cases
* Use require.NoError instead of assertNoError
* Add context to store interface
* Update logging to be less verbose
* Use io instead of ioutil package
* Simplify restore by using object ID
* Accommodate repository.NewWriter signature change
* Improve tests to increase code coverage
* Spelling and error string fixes
* Address lint errors
Co-authored-by: Nick <nick@kasten.io>
Protect filesystem subtrees from concurrent manipulation during critical sections
if engine actions are called asynchronously. This change provides coordination
between the `Snapshotter` and the `FileWriter`. For example, the `FileWriter`
should be blocked from perturbing the same directory tree if a
Gather-Snapshot is taking place along that tree simultaneously.
This will ensure the fingerprint data accumulated during the `Gather` phase
will correspond unambiguously to the data included in the snapshot.
Extend build flags to kopia snapshotter
This package now imports fswalker which can only be built for
darwin,amd64 or linux,amd64
Add metadata R/W locking for asynchronous accesses to robustness engine metadata.
Remove the index from the Store interface and maintain it only in Checker, where it's used.
* linter: upgraded to 1.33, disabled some linters
* lint: fixed 'errorlint' errors
This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.
Verified that there are no exceptions for errorlint linter which
guarantees that.
* lint: fixed or suppressed wrapcheck errors
* lint: nolintlint and misc cleanups
Co-authored-by: Julio López <julio+gh@kasten.io>
Remove unnecessary intermediate variables.
Send SIGTERM instead of SIGKILL to terminate child kopia server process.
Set Pdeathsig on Linux for child kopia server process.
Trivial: reduce scope of hostFioDataPathStr variable.
Trivial: rename local variable.
Trivial: Use log.Fatalln instead of log + exit(1).
Improve error message in robustness test to tell apart failure cause.
* Robustness engine actions with stats and logging
- Add actions to robustness engine
- Actions wrap other functional behavior and serve as a common interface for collecting stats
- Add stats for the engine, both per run and cumulative over time
- Add a log for actions that the engine has executed
- Add recovery logic to re-sync snapshot metadata after a possible failed engine run (e.g. if metadata wasn't properly persisted).
Current built-in actions:
- snapshot root directory
- restore random snapshot ID into a target restore path
- delete a random snapshot ID
- run GC
- write random files to the local data directory
- delete a random subdirectory under the local data directory
- delete files in a directory
- restore a snapshot ID into the local data directory
Actions are executed according to a set of options, which dictate the relative probabilities of picking a given action, along with ranges for action-specific parameters that can be randomized.
* [Robustness] Fix for kopia runner and custom work dir
Apply fix similar to #293 for the robustness kopia runner.
Add control for runner working directory.
