Commit Graph

36 Commits

Author SHA1 Message Date
Jarek Kowalski
fa7976599c repo: refactored repository interfaces (#780)
- `repo.Repository` is now read-only and only has methods that can be supported over kopia server
- `repo.RepositoryWriter` has read-write methods that can be supported over kopia server
- `repo.DirectRepository` is read-only and contains all methods of `repo.Repository` plus some low-level methods for data inspection
- `repo.DirectRepositoryWriter` contains write methods for `repo.DirectRepository`

- `repo.Reader` removed and merged with `repo.Repository`
- `repo.Writer` became `repo.RepositoryWriter`
- `*repo.DirectRepository` struct became `repo.DirectRepository`
  interface

Getting `{Direct}RepositoryWriter` requires using `NewWriter()` or `NewDirectWriter()` on a read-only repository and multiple simultaneous writers are supported at the same time, each writing to their own indexes and pack blobs.

`repo.Open` returns `repo.Repository` (which is also `repo.RepositoryWriter`).

* content: removed implicit flush on content manager close
* repo: added tests for WriteSession() and implicit flush behavior
* invalidate manifest manager after write session

* cli: disable maintenance in 'kopia server start'
  Server will close the repository before completing.

* repo: unconditionally close RepositoryWriter in {Direct,}WriteSession
* repo: added panic in case somebody tries to create RepositoryWriter after closing repository
  - used atomic to manage SharedManager.closed

* removed stale example
* linter: fixed spurious failures

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-01-20 11:41:47 -08:00
Jarek Kowalski
5e8e175cfa repo: refactored read/write methods of repo.Repository (#749)
Reader methods go to repo.Reader and write methods go to repo.Writer
Switched usage to new interfaces based on linter errors.
2021-01-04 21:33:12 -08:00
Jarek Kowalski
e03971fc59 Upgraded linter to v1.33.0 (#734)
* linter: upgraded to 1.33, disabled some linters

* lint: fixed 'errorlint' errors

This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.

Verified that there are no exceptions for errorlint linter which
guarantees that.

* lint: fixed or suppressed wrapcheck errors

* lint: nolintlint and misc cleanups

Co-authored-by: Julio López <julio+gh@kasten.io>
2020-12-21 22:39:22 -08:00
Jarek Kowalski
0758a92c58 restore: improved user experience (#644)
* restore: improved user experience

* 'snapshot restore' is now the same as 'restore' and both will
  support restoring by manifest ID, root ID or root ID + subdirectory

* added support for restoring individual files

* implemented PR feedback and refactored object ID parsing

Moving helpers inside the snapshot/ package helped clean up the code
a lot.
2020-09-28 22:57:24 -07:00
Jarek Kowalski
ff6a414ec5 cli: When listing directory that had errors, print error summary at the end. (#643)
Can be disabled with `--no-error-summary`.
Quick demo: https://asciinema.org/a/2rma0sx2mD6HoIPy6VL0QEFeP

Also refactored fs.Directory to provide Summary optionally.
2020-09-25 09:06:41 -07:00
Jarek Kowalski
6cb9b8fa4f repo: refactored public API (#318)
* This is 99% mechanical:

Extracted repo.Repository interface that only exposes high-level object and manifest management methods, but not blob nor content management.

Renamed old *repo.Repository to *repo.DirectRepository

Reviewed codebase to only depend on repo.Repository as much as possible, but added way for low-level CLI commands to use DirectRepository.

* PR fixes
2020-03-26 08:04:01 -07:00
Jarek Kowalski
9b68a631e6 Highlight snapshot errors in the UI and CLI (#376)
* upload: exposed numFailed and failedEntries on directory summary

* cli: better present snapshot errors

* htmlui: display snapshot errors
2020-03-22 14:18:47 -07:00
Jarek Kowalski
6217df1a87 lint: switched to 1.21 and fixed a ton of whitespace issues discovered
by new wsl linter
2019-11-26 06:49:49 -08:00
Jarek Kowalski
72520029b0 golangci-lint: added more linters
Also fixed pre-existing lint errors.
2019-06-02 22:56:57 -07:00
Jarek Kowalski
63303904e1 switched remaining fmt.Errorf to errors.Wrap() 2019-06-01 10:57:05 -07:00
Jarek Kowalski
0c41d41276 Fixed up paths after merge 2019-05-27 15:48:39 -07:00
Jarek Kowalski
4edd0a13bb cli: swapped object ID and file name in ls output 2018-11-03 11:35:31 -07:00
Jarek Kowalski
242d012a54 removed fs.EntryMetadata and replaced fs.Entry with os.FileInfo + small additions 2018-11-03 11:28:06 -07:00
Jarek Kowalski
327d8317d8 refactored repo/ into separate github.com/kopia/repo/ git repository 2018-10-26 20:40:57 -07:00
Jarek Kowalski
c999c0fba8 moved fs/repofs to snapshot/snapshotfs 2018-10-20 11:36:14 -07:00
Jarek Kowalski
f4573f089e removed 'object' subcommand and flattened children directly under app 2018-10-06 09:55:12 -07:00
Jarek Kowalski
3c26be391a added global hidden --timezone flag for changing the timezone in which the timestamps are displayed 2018-10-06 09:54:43 -07:00
Jarek Kowalski
91066f2469 reorganized low-level repository packages by moving them all under kopia/kopia/repo/ 2018-08-30 22:01:05 -07:00
Jarek Kowalski
6484a55d7c removed policy.Manager and snapshot.Manager and switched to package-level function interface instead 2018-08-30 06:47:58 -07:00
Jarek Kowalski
9060e7d310 moved repository VFS nodes into fs/repofs/ package 2018-08-29 22:56:23 -07:00
Jarek Kowalski
f869996a1c aligned 'ls' output 2018-06-06 21:38:32 -07:00
Jarek Kowalski
e36d02c59b emit per-dir incomplete reason
store entire rootEntry on snapshot nmanifest
format 'snapshot list' consistently for both root and nested items
2018-05-19 22:00:14 -07:00
Jarek Kowalski
96d27c2293 improved block list and index format 2018-05-17 22:41:54 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
453bab3560 fixed some lint errors, mostly dead code and missing error checks 2018-03-19 12:26:28 -07:00
Jarek Kowalski
d3b854ef8e optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything 2017-11-23 15:55:56 -08:00
Jarek Kowalski
c3f31d744f removed stuttering 2017-10-14 16:38:22 -07:00
Jarek Kowalski
31e5bce1cb moved object management to kopia/kopia/object 2017-10-14 16:38:22 -07:00
Jarek Kowalski
32c603362c removed support for inline object IDs now that packs are supported everywhere 2017-09-16 09:27:32 -07:00
Jarek Kowalski
82657f957d extracted timeFormat constant for printing dates in CLI 2017-09-05 18:21:17 -07:00
Jarek Kowalski
76b712c2fb moved creation of repofs entries to snapshot.Manager 2017-09-04 20:47:51 -07:00
Jarek Kowalski
371fc15694 merged 'repofs' into 'snapshot' package 2017-09-04 20:26:25 -07:00
Jarek Kowalski
cfbef13aff Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4 removed 'client' package and merged with 'repo', names still not final 2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc replaced use of Vault with Repository 2017-08-01 08:29:41 +02:00
Jarek Kowalski
4a878738a0 moved main executable package to kopia/kopia, moved cli support to kopia/kopia/cli 2017-07-17 15:20:32 +02:00