Commit Graph

668 Commits

Author SHA1 Message Date
Jarek Kowalski
08d62d60f1 revamped progress bar by including hashing and uploading part separately 2018-09-14 11:56:48 -07:00
Jarek Kowalski
636837317b added 'block verify' subcommand 2018-09-13 19:33:12 -07:00
Jarek Kowalski
a922c2b568 s3: fixed off-by-one error in s3_storage.GetBlock(), added more tests
added progress reporting to s3 provider
2018-09-13 19:33:12 -07:00
Jarek Kowalski
69782d2abe refactored cache commands 2018-09-13 19:33:12 -07:00
Jarek Kowalski
906b8eaee3 plumbed through context to all manifest manager methods and switched to lazy initialization 2018-09-13 15:34:29 -07:00
Jarek Kowalski
f2c7c29a57 fixed travis build 2018-09-10 19:31:57 -07:00
Jarek Kowalski
3d0ab23c03 switched release process to using goreleaser 2018-09-10 18:55:10 -07:00
Jarek Kowalski
93e03635fe added dist/ to .gitignore 2018-09-09 18:24:12 -07:00
Jarek Kowalski
3b82eadaa6 lint: disabled gosec linter until issues are fixed 2018-09-09 18:23:47 -07:00
Jarek Kowalski
59a3f4beec switched to stable version of gopkg.in/cheggaaa/pb.v1 2018-09-09 16:17:08 -07:00
Jarek Kowalski
4cf73fdd0f moved storagetesting to kopia/internal 2018-09-08 11:08:51 -07:00
Jarek Kowalski
1b014c875a simplified repository API password handling.
completely rewrote password storage:

- by default passwords are kept in OS-specific keyring (Keychain on macOS,
Windows Credentials Manager on Windows), which can be optionally disabled
to store password in a local file.

- on Linux keychain is disabled by default (does not work reliably
in terminal sessions), but can be enabled using command-line flag.
2018-09-07 21:34:31 -07:00
Jarek Kowalski
1bbc169c0d added missing package godoc, fixed test paths 2018-08-31 19:12:58 -07:00
Jarek Kowalski
6d2d663cac moved CLI storage support to cli/ 2018-08-30 22:08:55 -07:00
Jarek Kowalski
91066f2469 reorganized low-level repository packages by moving them all under kopia/kopia/repo/ 2018-08-30 22:01:05 -07:00
Jarek Kowalski
6484a55d7c removed policy.Manager and snapshot.Manager and switched to package-level function interface instead 2018-08-30 06:47:58 -07:00
Jarek Kowalski
a44c9554c0 refactored upload to separate package 2018-08-30 06:21:40 -07:00
Jarek Kowalski
3ef4593b03 refactored policy management into separate policy/ package 2018-08-29 23:08:40 -07:00
Jarek Kowalski
9060e7d310 moved repository VFS nodes into fs/repofs/ package 2018-08-29 22:56:23 -07:00
Jarek Kowalski
0f20965143 added 'blockindex recover' tests 2018-08-25 16:30:11 -07:00
Jarek Kowalski
5dd755fcec added support for index recovery from pack files, work in progress 2018-08-25 16:29:32 -07:00
Jarek Kowalski
98aa71c02a svg image, deleted png images 2018-08-20 22:30:16 -07:00
Jarek Kowalski
a8a609c816 svg image 2018-08-20 22:26:27 -07:00
Jarek Kowalski
30c1c05aa0 updated readme and added logo 2018-08-20 22:16:30 -07:00
Jarek Kowalski
8850c69f2e updated readme and added logo 2018-08-20 22:08:08 -07:00
Jarek Kowalski
de53519c66 Fixed a race condition between two manifest
managers, where the first one compacts blocks
and second one is in the middle of listing
and loading all blocks.

This caused flakiness in the stress test.
2018-08-20 20:53:52 -07:00
Jarek Kowalski
6587ade1f9 cli: added 'policy edit' command which launches an editor and allows user to edit policy JSON directly 2018-08-20 19:07:44 -07:00
Jarek Kowalski
f79571b8b3 upload: use runtime.NumCPU() for parallel upload 2018-08-19 11:48:55 -10:00
Jarek Kowalski
864b763cc0 progress: output the # of files in each directory 2018-08-19 11:48:55 -10:00
Jarek Kowalski
1ff003c820 added internal lib for gitignore-style pattern matching
added 'ignorefs' which transparently hides all files according to .kopiaignore, which is very similar to .gitignore
2018-08-19 11:48:55 -10:00
Jarek Kowalski
5d73d18347 removed Parent from fs.Entry 2018-08-19 11:32:17 -10:00
Jarek Kowalski
5386370906 emit snapshot size 2018-08-04 09:57:10 -07:00
Jarek Kowalski
db6991cb1a switched to stable gometalinter.v2 2018-07-27 18:37:17 -07:00
Jarek Kowalski
9a3a2da5fa fixed stdout output of 'expire' 2018-07-27 18:28:58 -07:00
Jarek Kowalski
8fc77bd5b1 changed repository example to use filesystem instead of GCS 2018-07-22 12:16:44 -07:00
Jarek Kowalski
9543c85e5a introduced side-effecting import 'kopia/kopia/storage/providers' which registers all storage providers, by default no storage providers are dependencies of 'repo' 2018-07-22 12:09:42 -07:00
Jarek Kowalski
be32f6b717 enabled more comprehensive repository stress test after resolving remaining races:
skipped stress tests during short testing
changed default make target to run short tests
2018-07-20 18:48:00 -08:00
Jarek Kowalski
00b18b4da4 switched to consistent listing of block indexes 2018-07-20 18:36:54 -08:00
Jarek Kowalski
14feae8193 replaced silent return with t.Skip() in s3_storage_test.go 2018-07-20 18:11:28 -08:00
Jarek Kowalski
bc835955d5 added utility functions to fetch all matching list results without a callback until a self-consistent snapshot has been reached 2018-07-20 18:11:28 -08:00
Jarek Kowalski
306ab5888d changed source manager to handle remote hosts differently 2018-07-19 15:03:48 -08:00
Jarek Kowalski
8b00bdb284 fixed 'coverage' makefile target 2018-07-19 15:02:45 -08:00
Jarek Kowalski
14471f124d disable s3 test if the endpoint is not available 2018-07-19 15:02:16 -08:00
Jarek Kowalski
6af6914b40 reject invalid offset/length in map storage 2018-07-19 15:01:53 -08:00
Jarek Kowalski
a91c46e9b4 added block cache tests 2018-07-19 15:01:17 -08:00
Jarek Kowalski
b2b34c1dea reacted to a change in gometalinter that failed the build 2018-07-10 06:19:22 -07:00
Jarek Kowalski
09c80ed0e9 disabled repository stress 2018-07-09 22:17:11 -07:00
Jarek Kowalski
a299bdb6f9 fixed linter errors 2018-07-09 21:51:24 -07:00
Jarek Kowalski
ec66349eeb Fixed nasty bug where if the storage fails to write a block we would clobber pending block data in memory by encrypting it in place. This was happening a lot when backing up to cloud and closing the laptop whihle the upload was going on. Added regression test and a rudimentary mechanism for fault injection in storage. 2018-07-09 21:28:12 -07:00
Jarek Kowalski
801b6eb5eb cleaned up block manager to make enforcement of invariants easier 2018-07-09 21:28:12 -07:00