Commit Graph

  • ebd8b0a943 CLI improvements Jarek Kowalski 2018-06-04 22:55:57 -07:00
  • 416475a092 added RewriteBlock() support which can rewrite either deleted or non-deleted block to a new pack Jarek Kowalski 2018-06-04 22:06:27 -07:00
  • 438fa4a587 whenever two blocks in index have identical times, prefer the non-deleted one Jarek Kowalski 2018-06-04 17:45:31 -07:00
  • f6dcd3c155 remove old unused cached index blocks Jarek Kowalski 2018-06-02 10:51:32 -07:00
  • 02c5e90671 added new cli flags to 'storage list' Jarek Kowalski 2018-06-02 10:46:59 -07:00
  • 5ba913380e added support for soft-delete that preserves original pack ID/offset/length. removed legacy soft-delete support without this information Jarek Kowalski 2018-05-30 21:16:48 -07:00
  • 53db414ff7 Added simple analytics mechanism based on Google Analytics for tracking features usage and latency. Controlled on command line via --analytics-consent (defaults to asking user) Jarek Kowalski 2018-05-29 22:22:39 -07:00
  • 80a3bdfd4a fixed race condition when multiple kopia instances are downloading indexes in parallel Jarek Kowalski 2018-05-20 10:29:11 -07:00
  • e36d02c59b emit per-dir incomplete reason store entire rootEntry on snapshot nmanifest format 'snapshot list' consistently for both root and nested items Jarek Kowalski 2018-05-19 21:56:31 -07:00
  • 37566b0a05 output per-directory summary when uploading Jarek Kowalski 2018-05-19 07:04:42 -07:00
  • 7f6c08835e changed naming of retention tags to be more explanatory Jarek Kowalski 2018-05-18 20:03:50 -07:00
  • e6c8144b93 output pack block id in 'blockindex show active' Jarek Kowalski 2018-05-17 23:26:43 -07:00
  • 490fa6130b changed pack naming scheme to be p{random-128-bit} Jarek Kowalski 2018-05-17 23:00:24 -07:00
  • 96d27c2293 improved block list and index format Jarek Kowalski 2018-05-17 22:41:54 -07:00
  • ec779b14c7 refactored and cleaned up policy management, added retention tags to snapshot list Jarek Kowalski 2018-05-17 20:31:37 -07:00
  • c907580aaf when running 'snapshot create --all' ignore individual source error failures Jarek Kowalski 2018-05-15 20:54:45 -07:00
  • d6477377f9 changed how index compaction works: Jarek Kowalski 2018-05-15 20:09:56 -07:00
  • 7e6154f6f3 fixed progress bar Jarek Kowalski 2018-05-13 20:33:35 -07:00
  • ad6330fe58 disable test caching Jarek Kowalski 2018-05-09 22:26:25 -07:00
  • 6f7b3b8a80 fixed stress test failures due to expiration of a context Jarek Kowalski 2018-05-09 22:20:00 -07:00
  • 12c28e537d simplified ObjectID code and changed it to a string Jarek Kowalski 2018-05-09 22:19:36 -07:00
  • 04acb44669 cleanup Jarek Kowalski 2018-05-09 07:45:14 -07:00
  • 0ba37c4ccb run longer stress test on travis Jarek Kowalski 2018-05-09 07:42:12 -07:00
  • a30af40f7a added 1s sleep to fix flakiness in manifest_manager_test.go due to rounding timestamps down to 1s Jarek Kowalski 2018-05-09 06:28:06 -07:00
  • db0fd9ed28 added packindex.IsSubset() Jarek Kowalski 2018-05-06 16:37:35 -07:00
  • 9c57f22c0f simplified committed index block interface, speed up block block manager test by a lot Jarek Kowalski 2018-05-05 16:12:20 -07:00
  • 62ce401cd6 exposed block.Manager.ListBlocks() which only returns block IDs instead of full info Jarek Kowalski 2018-05-05 11:13:39 -07:00
  • 3a02ef24e0 retire block.ContentID and replace with string Jarek Kowalski 2018-05-05 10:34:39 -07:00
  • be3c4e725e added stress test Jarek Kowalski 2018-05-05 10:07:20 -07:00
  • 39aa586b80 fixed map storage data aliasing Jarek Kowalski 2018-05-05 09:50:04 -07:00
  • 1342fd910a switched to using custom block index designed for efficient lookup without having to parse it the block index stores tightly packed sorted block IDs Jarek Kowalski 2018-04-29 16:32:06 -07:00
  • 250ceff6fd removed support for pack index V1 Jarek Kowalski 2018-04-29 14:10:20 -07:00
  • 3464059031 cleanup Jarek Kowalski 2018-04-29 13:53:35 -07:00
  • 5ede755524 added pack index v2 Jarek Kowalski 2018-04-28 20:38:49 -07:00
  • 5912c628bb added copyPackIndex() Jarek Kowalski 2018-04-28 18:38:11 -07:00
  • 4428743a5b added --raw option to 'blockindex show' Jarek Kowalski 2018-04-28 17:30:08 -07:00
  • 726d33e0d5 added levelDB-based persistent block index Jarek Kowalski 2018-04-27 19:31:13 -07:00
  • c4c74586c6 refactored block manager so that loaded block index is separate from pending block index expanded block.Info to include additional details, removed packBlockInfo removed repackaging support for now, since it's greatly complicating the implementation Jarek Kowalski 2018-04-23 21:55:03 -07:00
  • f94fcea14b moved block cache to a subdirectory Jarek Kowalski 2018-04-23 19:56:01 -07:00
  • df7a9e217a added support for 'v1' pack format that stores each packed block individually encrypted as opposed to encrypting the entire pack Jarek Kowalski 2018-04-23 17:51:00 -07:00
  • 0f63f04c14 changed how pack block are staged to first add them as inline, then convert to a block as necessary and not the opposite Jarek Kowalski 2018-04-21 12:42:33 -07:00
  • ab4d2c7f8e store format version with the index, not used yet Jarek Kowalski 2018-04-21 10:36:30 -07:00
  • 52bcb6223f when loading active index, load 10 minutes extra to correct for clock skew and in-flight uploads Jarek Kowalski 2018-04-21 10:23:59 -07:00
  • 49dfc43ad3 example: stopped persisting credentials Jarek Kowalski 2018-04-20 21:31:21 -07:00
  • 8763c31db9 added examples of using repository API Jarek Kowalski 2018-04-15 20:12:02 -07:00
  • 7d08d9921b fail to initialize repository if the format block already exists Jarek Kowalski 2018-04-15 20:09:54 -07:00
  • 7159c2ed07 removed packInfo.isEmpty Jarek Kowalski 2018-04-14 19:22:03 -07:00
  • a2e8954d85 changed how deleted blocks are managed to allow indexes to be reloaded Jarek Kowalski 2018-04-14 19:05:19 -07:00
  • 14e38b59bf cleaned up how active block index is loaded Jarek Kowalski 2018-04-14 16:25:03 -07:00
  • 825f051515 added constraint that the block.ContentID prefix must be a single non-hex character or an empty string. Jarek Kowalski 2018-04-10 22:37:32 -07:00
  • 924e0a4594 renamed index proto types to IndexV1 Jarek Kowalski 2018-04-08 17:17:00 -07:00
  • 301f1d049a added strongly-typed block.ContentID and renamed StorageBlock to ContentBlockID Jarek Kowalski 2018-04-08 12:54:15 -07:00
  • 4566213f5d switched formatter to pass IV explicitly extracted packIndex interface and implemented protoPackIndex to wrap all index data manipulation Jarek Kowalski 2018-04-07 11:19:08 -07:00
  • d8201229d8 plumbed through ctx in storage.Storage APIs and all uses Jarek Kowalski 2018-04-01 20:35:07 -07:00
  • c702e15489 improved logging of hash cache misses Jarek Kowalski 2018-03-25 16:54:12 -07:00
  • 2b49a5df7b added per-directory summary Jarek Kowalski 2018-03-24 09:40:37 -07:00
  • e997e07b0c added support for summaries to JSON stream Jarek Kowalski 2018-03-24 09:07:59 -07:00
  • a608c2f443 tweaked snapshot list Jarek Kowalski 2018-03-20 09:49:38 -07:00
  • 1c4e0e1e47 removed legacy oid format Jarek Kowalski 2018-03-19 14:43:43 -07:00
  • 2e61cc5688 code simplification, enabled gocyclo linter Jarek Kowalski 2018-03-19 13:48:34 -07:00
  • 8d3b1df2fa fixed build break caused by using deprecated CloseWithError() Jarek Kowalski 2018-03-19 13:25:20 -07:00
  • fa5eae501a removed cleanup for now Jarek Kowalski 2018-03-19 11:56:16 -07:00
  • ab4fa93c92 remove cache directory on disconnect Jarek Kowalski 2018-03-19 10:57:52 -07:00
  • 453bab3560 fixed some lint errors, mostly dead code and missing error checks Jarek Kowalski 2018-03-16 21:26:35 -07:00
  • 4657f16cc8 CLI improvement for automatic logging and log rotation Jarek Kowalski 2018-03-15 09:39:57 -07:00
  • 47b1e09ba9 added default CLI commands Jarek Kowalski 2018-03-15 09:39:39 -07:00
  • 54aeea8ee6 fixed block list for inline blocks Jarek Kowalski 2018-03-14 19:27:52 -07:00
  • e1d5120727 additional logging, fixed manifest manager compaction Jarek Kowalski 2018-03-14 18:22:13 -07:00
  • 70ed5647e4 added parallel uploading support Jarek Kowalski 2018-03-12 18:05:50 -07:00
  • a50546de0e fixed previous manifest selection for incremental snapshots Jarek Kowalski 2018-03-12 16:50:01 -07:00
  • f4c9749246 additional logging for hash cache Jarek Kowalski 2018-03-12 16:49:45 -07:00
  • 0fa823976f changed upload order to post-order - we first preprocess all subdirectories of a given directory, followed by files Jarek Kowalski 2018-03-12 09:17:27 -07:00
  • eba1d94eff added invariant checks to block.Manager Jarek Kowalski 2018-03-10 18:10:22 -08:00
  • 14bda2aefc added auto-compaction of block indexes Jarek Kowalski 2018-03-04 18:46:51 -08:00
  • 772ef9a286 added 'ls' alias for 'snapshot list' Jarek Kowalski 2018-03-04 18:17:18 -08:00
  • 0b8655998c bump timeouts to fix travis flakiness Jarek Kowalski 2018-03-04 17:19:28 -08:00
  • 6707d01a22 switched local caching to use storage.Storage Jarek Kowalski 2018-03-04 16:58:20 -08:00
  • a9f8741e8e changed local caching to be based on virtual blocks instead of physical blocks Jarek Kowalski 2018-03-04 12:13:57 -08:00
  • 32ea28ba41 dream cache logging Jarek Kowalski 2018-02-26 19:38:00 -08:00
  • 6efeb8a8ff fixed deletion of LRU items in block cache Jarek Kowalski 2018-02-24 23:41:27 -08:00
  • 2a72a57dc8 cleanup Jarek Kowalski 2018-02-20 22:47:40 -08:00
  • 92330ac247 refactored policy manager to use SourceInfo Jarek Kowalski 2018-02-19 13:20:32 -08:00
  • 16e44cb37f renamed policy.Manager as snapshot.PolicyManager and policy.Policy as snapshot.Policy Jarek Kowalski 2018-02-19 10:59:15 -08:00
  • d97f49285f policy CLI and API cleanup Jarek Kowalski 2018-02-19 10:53:29 -08:00
  • e710dcdff3 additional integration tests Jarek Kowalski 2018-02-18 11:59:32 -08:00
  • a92ea34559 upgrade go to 1.10, try #2 Jarek Kowalski 2018-02-18 11:00:19 -08:00
  • bd06941b8e upgrade go to 1.10 Jarek Kowalski 2018-02-18 10:58:24 -08:00
  • 663fee1ef1 rudimentary end-to-end tests to be running as part of travis Jarek Kowalski 2018-02-18 10:52:15 -08:00
  • 986dc02ad9 ignore manifest entries marked as deleted Jarek Kowalski 2018-02-18 10:39:16 -08:00
  • b6ae16bfb6 changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs Jarek Kowalski 2018-02-16 18:18:47 -08:00
  • faa2625a5f revamped CLI help to hide most commands Jarek Kowalski 2018-02-15 19:49:51 -08:00
  • 01ffec69fc fixed output of 'storage list' for long block names Jarek Kowalski 2018-02-11 08:12:20 -08:00
  • 4c5377898f fixed handling of degenerate cases in 'block repack' Jarek Kowalski 2018-02-11 08:11:31 -08:00
  • a350991762 reorganized CLI packages to be close to storage providers Jarek Kowalski 2018-02-11 08:00:07 -08:00
  • 16095ae900 removed block prefixes in object manager, changed metadata prefix to lowercase m Jarek Kowalski 2018-02-10 19:47:30 -08:00
  • 06b4bec246 added 'storage sweep' subcommand which removes unreferenced storage blocks Jarek Kowalski 2018-02-10 12:50:21 -08:00
  • 4971671f3d added pretty printing of block indexes Jarek Kowalski 2018-02-10 12:07:13 -08:00
  • 4564bc704f breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks align pack blocks to 4096 bytes and insert random preamble Jarek Kowalski 2018-02-03 18:33:47 -08:00
  • 7e16471a21 added --flat option to filesystem and webdav providers Jarek Kowalski 2018-02-07 21:50:53 -08:00
  • df35ebad73 added low-level storage viewing subcommands Jarek Kowalski 2018-02-03 22:11:53 -08:00