Commit Graph

6 Commits

Author SHA1 Message Date
Jarek Kowalski
fa7976599c repo: refactored repository interfaces (#780)
- `repo.Repository` is now read-only and only has methods that can be supported over kopia server
- `repo.RepositoryWriter` has read-write methods that can be supported over kopia server
- `repo.DirectRepository` is read-only and contains all methods of `repo.Repository` plus some low-level methods for data inspection
- `repo.DirectRepositoryWriter` contains write methods for `repo.DirectRepository`

- `repo.Reader` removed and merged with `repo.Repository`
- `repo.Writer` became `repo.RepositoryWriter`
- `*repo.DirectRepository` struct became `repo.DirectRepository`
  interface

Getting `{Direct}RepositoryWriter` requires using `NewWriter()` or `NewDirectWriter()` on a read-only repository and multiple simultaneous writers are supported at the same time, each writing to their own indexes and pack blobs.

`repo.Open` returns `repo.Repository` (which is also `repo.RepositoryWriter`).

* content: removed implicit flush on content manager close
* repo: added tests for WriteSession() and implicit flush behavior
* invalidate manifest manager after write session

* cli: disable maintenance in 'kopia server start'
  Server will close the repository before completing.

* repo: unconditionally close RepositoryWriter in {Direct,}WriteSession
* repo: added panic in case somebody tries to create RepositoryWriter after closing repository
  - used atomic to manage SharedManager.closed

* removed stale example
* linter: fixed spurious failures

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-01-20 11:41:47 -08:00
Jarek Kowalski
e03971fc59 Upgraded linter to v1.33.0 (#734)
* linter: upgraded to 1.33, disabled some linters

* lint: fixed 'errorlint' errors

This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.

Verified that there are no exceptions for errorlint linter which
guarantees that.

* lint: fixed or suppressed wrapcheck errors

* lint: nolintlint and misc cleanups

Co-authored-by: Julio López <julio+gh@kasten.io>
2020-12-21 22:39:22 -08:00
Jarek Kowalski
9d7cf71a37 Logging flags (#674)
* logging: cleaned up stderr logging

- do not show module
- do not show timestamps by default (enable with --console-timestamps)

* logging: replaced most printStderr() with log.Info

* cli: additional logging cleanup
2020-10-10 10:48:37 -07:00
Jarek Kowalski
c242235a32 blob: added SetTime() method which may be optionally implemented by blob.Storage (#575)
cli: added --times option to 'repository sync'
2020-08-31 19:50:15 -07:00
Jarek Kowalski
1a8fcb086c Added endurance test which tests kopia over long time scale (#558)
Globally replaced all use of time with internal 'clock' package
which provides indirection to time.Now()

Added support for faking clock in Kopia via KOPIA_FAKE_CLOCK_ENDPOINT

logfile: squelch annoying log message

testenv: added faketimeserver which serves time over HTTP

testing: added endurance test which tests kopia over long time scale

This creates kopia repository and simulates usage of Kopia over multiple
months (using accelerated fake time) to trigger effects that are only
visible after long time passage (maintenance, compactions, expirations).

The test is not used part of any test suite yet but will run in
post-submit mode only, preferably 24/7.

testing: refactored internal/clock to only support injection when
'testing' build tag is present
2020-08-26 23:03:46 -07:00
Jarek Kowalski
505ab92e21 Support for repository sync (#522)
* blob: added DisplayName() method to blob.Storage

* cli: added 'kopia repo sync-to <provider>' which replicates BLOBs

Usage demo: https://asciinema.org/a/352299

Fixes #509

* implemented suggestion by Ciantic to fail sync if the destination repository is not compatible with the source

* cli: added 'kopia repo sync --must-exist'

This ensures that target repository is not empty, otherwise syncing to
an accidentally unmounted filesystem directory might copy everything
again.
2020-08-09 12:36:41 -07:00