Commit Graph

524 Commits

Author SHA1 Message Date
Jarek Kowalski
7159c2ed07 removed packInfo.isEmpty 2018-04-14 19:26:58 -07:00
Jarek Kowalski
a2e8954d85 changed how deleted blocks are managed to allow indexes to be reloaded 2018-04-14 19:26:58 -07:00
Jarek Kowalski
14e38b59bf cleaned up how active block index is loaded 2018-04-14 19:26:58 -07:00
Jarek Kowalski
825f051515 added constraint that the block.ContentID prefix must be a single non-hex character or an empty string.
added unit tests for packIndex
2018-04-14 14:47:05 -07:00
Jarek Kowalski
924e0a4594 renamed index proto types to IndexV1 2018-04-08 18:45:22 -07:00
Jarek Kowalski
301f1d049a added strongly-typed block.ContentID and renamed StorageBlock to ContentBlockID 2018-04-08 16:34:38 -07:00
Jarek Kowalski
4566213f5d switched formatter to pass IV explicitly
extracted packIndex interface and implemented protoPackIndex to wrap all index data manipulation
2018-04-08 12:20:32 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
c702e15489 improved logging of hash cache misses 2018-03-25 16:54:12 -07:00
Jarek Kowalski
2b49a5df7b added per-directory summary 2018-03-24 12:22:21 -07:00
Jarek Kowalski
e997e07b0c added support for summaries to JSON stream 2018-03-24 12:22:21 -07:00
Jarek Kowalski
a608c2f443 tweaked snapshot list 2018-03-24 12:22:21 -07:00
Jarek Kowalski
1c4e0e1e47 removed legacy oid format 2018-03-19 19:31:03 -07:00
Jarek Kowalski
2e61cc5688 code simplification, enabled gocyclo linter 2018-03-19 14:08:19 -07:00
Jarek Kowalski
8d3b1df2fa fixed build break caused by using deprecated CloseWithError() 2018-03-19 13:25:20 -07:00
Jarek Kowalski
fa5eae501a removed cleanup for now 2018-03-19 12:26:28 -07:00
Jarek Kowalski
ab4fa93c92 remove cache directory on disconnect 2018-03-19 12:26:28 -07:00
Jarek Kowalski
453bab3560 fixed some lint errors, mostly dead code and missing error checks 2018-03-19 12:26:28 -07:00
Jarek Kowalski
4657f16cc8 CLI improvement for automatic logging and log rotation 2018-03-16 22:21:57 -07:00
Jarek Kowalski
47b1e09ba9 added default CLI commands 2018-03-15 09:39:39 -07:00
Jarek Kowalski
54aeea8ee6 fixed block list for inline blocks 2018-03-14 19:27:52 -07:00
Jarek Kowalski
e1d5120727 additional logging, fixed manifest manager compaction 2018-03-14 18:23:02 -07:00
Jarek Kowalski
70ed5647e4 added parallel uploading support 2018-03-14 18:22:50 -07:00
Jarek Kowalski
a50546de0e fixed previous manifest selection for incremental snapshots 2018-03-12 16:50:01 -07:00
Jarek Kowalski
f4c9749246 additional logging for hash cache 2018-03-12 16:49:45 -07:00
Jarek Kowalski
0fa823976f changed upload order to post-order - we first preprocess all subdirectories of a given directory, followed by files 2018-03-12 09:17:27 -07:00
Jarek Kowalski
eba1d94eff added invariant checks to block.Manager 2018-03-10 18:10:22 -08:00
Jarek Kowalski
14bda2aefc added auto-compaction of block indexes 2018-03-04 18:52:06 -08:00
Jarek Kowalski
772ef9a286 added 'ls' alias for 'snapshot list' 2018-03-04 18:17:18 -08:00
Jarek Kowalski
0b8655998c bump timeouts to fix travis flakiness 2018-03-04 17:19:28 -08:00
Jarek Kowalski
6707d01a22 switched local caching to use storage.Storage 2018-03-04 16:58:20 -08:00
Jarek Kowalski
a9f8741e8e changed local caching to be based on virtual blocks instead of physical blocks 2018-03-04 12:13:57 -08:00
Jarek Kowalski
32ea28ba41 dream cache logging 2018-02-26 19:38:00 -08:00
Jarek Kowalski
6efeb8a8ff fixed deletion of LRU items in block cache 2018-02-24 23:41:27 -08:00
Jarek Kowalski
2a72a57dc8 cleanup 2018-02-20 22:47:40 -08:00
Jarek Kowalski
92330ac247 refactored policy manager to use SourceInfo 2018-02-19 13:20:32 -08:00
Jarek Kowalski
16e44cb37f renamed policy.Manager as snapshot.PolicyManager and policy.Policy as snapshot.Policy 2018-02-19 10:59:15 -08:00
Jarek Kowalski
d97f49285f policy CLI and API cleanup 2018-02-19 10:53:29 -08:00
Jarek Kowalski
e710dcdff3 additional integration tests 2018-02-18 11:59:32 -08:00
Jarek Kowalski
a92ea34559 upgrade go to 1.10, try #2 2018-02-18 11:00:19 -08:00
Jarek Kowalski
bd06941b8e upgrade go to 1.10 2018-02-18 10:58:24 -08:00
Jarek Kowalski
663fee1ef1 rudimentary end-to-end tests to be running as part of travis 2018-02-18 10:52:15 -08:00
Jarek Kowalski
986dc02ad9 ignore manifest entries marked as deleted 2018-02-18 10:39:16 -08:00
Jarek Kowalski
b6ae16bfb6 changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs 2018-02-16 18:18:47 -08:00
Jarek Kowalski
faa2625a5f revamped CLI help to hide most commands 2018-02-15 19:49:51 -08:00
Jarek Kowalski
01ffec69fc fixed output of 'storage list' for long block names 2018-02-11 08:12:20 -08:00
Jarek Kowalski
4c5377898f fixed handling of degenerate cases in 'block repack' 2018-02-11 08:11:31 -08:00
Jarek Kowalski
a350991762 reorganized CLI packages to be close to storage providers 2018-02-11 08:00:07 -08:00
Jarek Kowalski
16095ae900 removed block prefixes in object manager, changed metadata prefix to lowercase m 2018-02-10 19:47:30 -08:00
Jarek Kowalski
06b4bec246 added 'storage sweep' subcommand which removes unreferenced storage blocks 2018-02-10 13:07:10 -08:00