Commit Graph

3660 Commits

Author SHA1 Message Date
Jarek Kowalski
393e7a8d59 disabled ANSI escape sequences when logging on windows 2018-01-04 17:45:15 -08:00
Jarek Kowalski
f9f2c54993 added command to reconfigure caching on a repository that's already connected 2018-01-04 17:44:58 -08:00
Jarek Kowalski
7af93182a4 changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs 2018-01-02 19:51:03 -08:00
Jarek Kowalski
10ca234633 switched to using zerolog for logging, by default log level=info, can be set to debug on commandline 2018-01-01 19:12:35 -08:00
Jarek Kowalski
ef2ac1b5eb dead code 2018-01-01 17:16:51 -08:00
Jarek Kowalski
9bd93ee787 stale comment 2018-01-01 17:12:56 -08:00
Jarek Kowalski
98070064c8 index block list caching optimization - only load last 2 full compactions when scanning for active index blocks, this prevents the scan time from growing indefinitely 2018-01-01 17:11:41 -08:00
Jarek Kowalski
6dda6ca5e5 prefix each index block name with an inverse timestamp, that decreases over time an can be lexicographically sorted to produce newest blocks first when listing by prefix P 2017-12-31 16:16:10 -08:00
Jarek Kowalski
596c2b122a disk cache: remove list after a write
fix close
2017-12-31 16:15:58 -08:00
Jarek Kowalski
51c1c77fb8 map initialization tweak 2017-11-27 19:40:23 -08:00
Jarek Kowalski
7f73ca2e52 added missing comments, cleanup 2017-11-27 19:34:07 -08:00
Jarek Kowalski
c288daef04 eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead 2017-11-27 19:15:02 -08:00
Jarek Kowalski
a644f2caa6 added 'manifest show' subcommand 2017-11-27 18:49:46 -08:00
Jarek Kowalski
30c11dc926 refactored block manager to support on-disk caching 2017-11-27 18:07:16 -08:00
Jarek Kowalski
a0cc09b673 removed caching storage 2017-11-27 18:06:28 -08:00
Jarek Kowalski
a700508048 changed API for manifest manager, added filtering, sorting and better output to 'manifest ls' 2017-11-27 18:06:28 -08:00
Jarek Kowalski
1968e2fa10 fix support for encrypted blocks with dash suffix 2017-11-27 18:05:44 -08:00
Jarek Kowalski
eebb04c0cb added 'manifest ls' subcommand 2017-11-27 18:05:44 -08:00
Jarek Kowalski
6e83ad0b9b moved policies to separate package
refactored policy management into policy.Manager that uses manifest.Manager internally
2017-11-27 18:05:32 -08:00
Jarek Kowalski
de02941442 plumbed through key time in map storage 2017-11-23 16:43:34 -08:00
Jarek Kowalski
d3b854ef8e optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything 2017-11-23 15:55:56 -08:00
Jarek Kowalski
29ed48f734 manifest manager compaction 2017-11-23 10:49:06 -08:00
Jarek Kowalski
74195e316e switched buzhash implementation to github.com/silvasur/buzhash, turns out previous library was used incorrectly, this one has simpler API, produces different split points 2017-11-14 22:45:15 -08:00
Jarek Kowalski
15602e98f5 output version and build info 2017-11-12 22:43:36 -08:00
Jarek Kowalski
40a367f79e added 'block rm' subcommand 2017-11-12 22:43:21 -08:00
Jarek Kowalski
585b6ddcc1 additional test for race involving blockmanager.delete 2017-11-12 20:43:33 -08:00
Jarek Kowalski
c0d71714e1 fixed a case where we would write a pack entry not corresponding to any block 2017-11-12 18:52:32 -08:00
Jarek Kowalski
50398692fe added support for block manager delete 2017-11-11 10:06:27 -08:00
Jarek Kowalski
5b287815ad add snapshot manifest to manifest.Manager at the end of upload 2017-11-10 20:18:57 -08:00
Jarek Kowalski
9c5c51dda7 deployment work 2017-11-08 22:23:46 -08:00
Jarek Kowalski
8982c985c9 travis creds 2017-11-08 22:09:06 -08:00
Jarek Kowalski
4967f6d406 fixed releasing 2017-11-08 21:10:26 -08:00
Jarek Kowalski
032e3cb737 merged 'format' and 'repo' blocks into 'kopia.repository' 2017-11-08 20:09:06 -08:00
Jarek Kowalski
2f25d8b9e8 initial manifest manager, not used yet 2017-11-08 18:58:41 -08:00
Jarek Kowalski
48a88eee09 removed dead code from metadata manage 2017-11-04 09:36:00 -07:00
Jarek Kowalski
113b1f55c7 cleanup 2017-10-29 19:38:07 -07:00
Jarek Kowalski
7c41f45e9f extracted block formatting options into a struct under 'block' 2017-10-29 13:00:24 -07:00
Jarek Kowalski
49a806154a code reorg 2017-10-29 12:03:49 -07:00
Jarek Kowalski
677f0535ed split repo connect/initialize/open into separate files 2017-10-29 11:48:45 -07:00
Jarek Kowalski
e98a3b3325 tweaks 2017-10-29 11:45:53 -07:00
Jarek Kowalski
f70d2af03d cleanup 2017-10-29 11:42:11 -07:00
Jarek Kowalski
0f6c569435 refactored repository status 2017-10-29 10:35:06 -07:00
Jarek Kowalski
b6575a97ba moved metadata manager to 'metadata' package and refactored initialization sequence 2017-10-29 10:29:22 -07:00
Jarek Kowalski
769c3739bd refactored repository config out of metadata manager 2017-10-29 08:44:53 -07:00
Jarek Kowalski
ac94dd1705 refactored repo.Connect to avoid calling connect() as most of what it does is not actually needed 2017-10-28 23:40:39 -07:00
Jarek Kowalski
3498f78d91 dead code cleanup 2017-10-28 22:50:11 -07:00
Jarek Kowalski
e76f1f8a72 refactored various show command into one 2017-10-24 21:59:40 -07:00
Jarek Kowalski
a48d54f00b added 'block show' similar to 'object show', but operating on virtual blocks 2017-10-24 21:15:22 -07:00
Jarek Kowalski
b4e6a70e09 renamed 'repo optimize' to 'block reindex', added repacking method and unit tests 2017-10-24 21:05:59 -07:00
Jarek Kowalski
2aa44eb454 added listing group blocks and summary to 'block list' 2017-10-17 19:24:16 -07:00