Commit Graph

8 Commits

Author SHA1 Message Date
Jarek Kowalski
6ce6a47d00 cli: fixed flags for blockindex subcommands 2019-01-12 11:19:38 -08:00
Jarek Kowalski
327d8317d8 refactored repo/ into separate github.com/kopia/repo/ git repository 2018-10-26 20:40:57 -07:00
Jarek Kowalski
61bdd33439 block: added compaction options to exclude deleted blocks and force compaction of large blocks 2018-09-15 17:43:00 -07:00
Jarek Kowalski
d6477377f9 changed how index compaction works:
instead of having time-based naming, block manager will perform occasional compaction at startup time and delete unwanted blocks

The protocol is safe from concurrency standpoint and provides eventual consistency.
We only delete blocks after creating compacted blocks.
We retry loading index until (any) consistent index is fetched (possibly from cache) and all underlying blocks are also available, not necessarily the latest ones.

TODO - we need to periodically snapshot block index contents, so that if we have a bug somewhere in compaction code, we have a way of restoring working indexes.
2018-05-15 20:10:50 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
453bab3560 fixed some lint errors, mostly dead code and missing error checks 2018-03-19 12:26:28 -07:00
Jarek Kowalski
d3b854ef8e optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything 2017-11-23 15:55:56 -08:00
Jarek Kowalski
e76f1f8a72 refactored various show command into one 2017-10-24 21:59:40 -07:00