mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-03-14 12:16:46 -04:00
Code Issues Packages Projects Releases Wiki Activity
524 Commits 4 Branches 88 Tags
7159c2ed07ba5cfa67255b35b30df641dafb2d37
Commit Graph

50 Commits

Author SHA1 Message Date
Jarek Kowalski
7159c2ed07 removed packInfo.isEmpty 2018-04-14 19:26:58 -07:00
Jarek Kowalski
a2e8954d85 changed how deleted blocks are managed to allow indexes to be reloaded 2018-04-14 19:26:58 -07:00
Jarek Kowalski
14e38b59bf cleaned up how active block index is loaded 2018-04-14 19:26:58 -07:00
Jarek Kowalski
825f051515 added constraint that the block.ContentID prefix must be a single non-hex character or an empty string. 
added unit tests for packIndex
 2018-04-14 14:47:05 -07:00
Jarek Kowalski
924e0a4594 renamed index proto types to IndexV1 2018-04-08 18:45:22 -07:00
Jarek Kowalski
301f1d049a added strongly-typed block.ContentID and renamed StorageBlock to ContentBlockID 2018-04-08 16:34:38 -07:00
Jarek Kowalski
4566213f5d switched formatter to pass IV explicitly 
extracted packIndex interface and implemented protoPackIndex to wrap all index data manipulation
 2018-04-08 12:20:32 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
1c4e0e1e47 removed legacy oid format 2018-03-19 19:31:03 -07:00
Jarek Kowalski
453bab3560 fixed some lint errors, mostly dead code and missing error checks 2018-03-19 12:26:28 -07:00
Jarek Kowalski
4657f16cc8 CLI improvement for automatic logging and log rotation 2018-03-16 22:21:57 -07:00
Jarek Kowalski
54aeea8ee6 fixed block list for inline blocks 2018-03-14 19:27:52 -07:00
Jarek Kowalski
e1d5120727 additional logging, fixed manifest manager compaction 2018-03-14 18:23:02 -07:00
Jarek Kowalski
eba1d94eff added invariant checks to block.Manager 2018-03-10 18:10:22 -08:00
Jarek Kowalski
14bda2aefc added auto-compaction of block indexes 2018-03-04 18:52:06 -08:00
Jarek Kowalski
6707d01a22 switched local caching to use storage.Storage 2018-03-04 16:58:20 -08:00
Jarek Kowalski
a9f8741e8e changed local caching to be based on virtual blocks instead of physical blocks 2018-03-04 12:13:57 -08:00
Jarek Kowalski
32ea28ba41 dream cache logging 2018-02-26 19:38:00 -08:00
Jarek Kowalski
6efeb8a8ff fixed deletion of LRU items in block cache 2018-02-24 23:41:27 -08:00
Jarek Kowalski
b6ae16bfb6 changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs 2018-02-16 18:18:47 -08:00
Jarek Kowalski
4c5377898f fixed handling of degenerate cases in 'block repack' 2018-02-11 08:11:31 -08:00
Jarek Kowalski
16095ae900 removed block prefixes in object manager, changed metadata prefix to lowercase m 2018-02-10 19:47:30 -08:00
Jarek Kowalski
06b4bec246 added 'storage sweep' subcommand which removes unreferenced storage blocks 2018-02-10 13:07:10 -08:00
Jarek Kowalski
4564bc704f breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks 
align pack blocks to 4096 bytes and insert random preamble
 2018-02-10 13:07:10 -08:00
Jarek Kowalski
7833626c4a added error handling when unable to load block index 2018-01-24 11:44:26 +00:00
Jarek Kowalski
d7e1276a48 cleaned up storage.Storage API and documentation 2018-01-10 19:13:09 -08:00
Jarek Kowalski
7af93182a4 changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs 2018-01-02 19:51:03 -08:00
Jarek Kowalski
10ca234633 switched to using zerolog for logging, by default log level=info, can be set to debug on commandline 2018-01-01 19:12:35 -08:00
Jarek Kowalski
ef2ac1b5eb dead code 2018-01-01 17:16:51 -08:00
Jarek Kowalski
9bd93ee787 stale comment 2018-01-01 17:12:56 -08:00
Jarek Kowalski
98070064c8 index block list caching optimization - only load last 2 full compactions when scanning for active index blocks, this prevents the scan time from growing indefinitely 2018-01-01 17:11:41 -08:00
Jarek Kowalski
6dda6ca5e5 prefix each index block name with an inverse timestamp, that decreases over time an can be lexicographically sorted to produce newest blocks first when listing by prefix P 2017-12-31 16:16:10 -08:00
Jarek Kowalski
596c2b122a disk cache: remove list after a write 
fix close
 2017-12-31 16:15:58 -08:00
Jarek Kowalski
51c1c77fb8 map initialization tweak 2017-11-27 19:40:23 -08:00
Jarek Kowalski
30c11dc926 refactored block manager to support on-disk caching 2017-11-27 18:07:16 -08:00
Jarek Kowalski
a700508048 changed API for manifest manager, added filtering, sorting and better output to 'manifest ls' 2017-11-27 18:06:28 -08:00
Jarek Kowalski
1968e2fa10 fix support for encrypted blocks with dash suffix 2017-11-27 18:05:44 -08:00
Jarek Kowalski
de02941442 plumbed through key time in map storage 2017-11-23 16:43:34 -08:00
Jarek Kowalski
d3b854ef8e optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything 2017-11-23 15:55:56 -08:00
Jarek Kowalski
585b6ddcc1 additional test for race involving blockmanager.delete 2017-11-12 20:43:33 -08:00
Jarek Kowalski
c0d71714e1 fixed a case where we would write a pack entry not corresponding to any block 2017-11-12 18:52:32 -08:00
Jarek Kowalski
50398692fe added support for block manager delete 2017-11-11 10:06:27 -08:00
Jarek Kowalski
7c41f45e9f extracted block formatting options into a struct under 'block' 2017-10-29 13:00:24 -07:00
Jarek Kowalski
b4e6a70e09 renamed 'repo optimize' to 'block reindex', added repacking method and unit tests 2017-10-24 21:05:59 -07:00
Jarek Kowalski
42a560fbab block.Manager: 
Maintaining per-group block index
Added block.Manager.ListGroupBlocks() API + tests
 2017-10-16 19:19:51 -07:00
Jarek Kowalski
44f8940a8f removed support for object ID prefixes 2017-10-14 16:38:27 -07:00
Jarek Kowalski
59bb3ef943 additional test for block manager 2017-10-14 16:38:27 -07:00
Jarek Kowalski
a715522c34 improved block stats output 2017-10-14 16:38:27 -07:00
Jarek Kowalski
f33ae14573 renamed 'blob.Storage' to 'storage.Storage' 2017-10-14 16:38:09 -07:00
Jarek Kowalski
cc3e590136 moved block manager and formatter to a separate package 2017-10-09 06:42:30 -07:00