Commit Graph

  • 10ca234633 switched to using zerolog for logging, by default log level=info, can be set to debug on commandline Jarek Kowalski 2018-01-01 19:12:35 -08:00
  • ef2ac1b5eb dead code Jarek Kowalski 2018-01-01 17:16:51 -08:00
  • 9bd93ee787 stale comment Jarek Kowalski 2018-01-01 17:12:56 -08:00
  • 98070064c8 index block list caching optimization - only load last 2 full compactions when scanning for active index blocks, this prevents the scan time from growing indefinitely Jarek Kowalski 2018-01-01 17:10:32 -08:00
  • 6dda6ca5e5 prefix each index block name with an inverse timestamp, that decreases over time an can be lexicographically sorted to produce newest blocks first when listing by prefix P Jarek Kowalski 2017-12-31 13:52:25 -08:00
  • 596c2b122a disk cache: remove list after a write fix close Jarek Kowalski 2017-12-31 11:19:49 -08:00
  • 51c1c77fb8 map initialization tweak Jarek Kowalski 2017-11-27 19:40:23 -08:00
  • 7f73ca2e52 added missing comments, cleanup Jarek Kowalski 2017-11-27 19:34:07 -08:00
  • c288daef04 eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead Jarek Kowalski 2017-11-27 19:15:02 -08:00
  • a644f2caa6 added 'manifest show' subcommand Jarek Kowalski 2017-11-27 18:49:33 -08:00
  • 30c11dc926 refactored block manager to support on-disk caching Jarek Kowalski 2017-11-27 18:07:16 -08:00
  • a0cc09b673 removed caching storage Jarek Kowalski 2017-11-26 10:38:55 -08:00
  • a700508048 changed API for manifest manager, added filtering, sorting and better output to 'manifest ls' Jarek Kowalski 2017-11-26 10:12:03 -08:00
  • 1968e2fa10 fix support for encrypted blocks with dash suffix Jarek Kowalski 2017-11-23 23:21:16 -08:00
  • eebb04c0cb added 'manifest ls' subcommand Jarek Kowalski 2017-11-23 23:15:24 -08:00
  • 6e83ad0b9b moved policies to separate package refactored policy management into policy.Manager that uses manifest.Manager internally Jarek Kowalski 2017-11-23 22:21:00 -08:00
  • de02941442 plumbed through key time in map storage Jarek Kowalski 2017-11-23 16:43:34 -08:00
  • d3b854ef8e optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything Jarek Kowalski 2017-11-23 15:55:56 -08:00
  • 29ed48f734 manifest manager compaction Jarek Kowalski 2017-11-23 10:49:06 -08:00
  • 74195e316e switched buzhash implementation to github.com/silvasur/buzhash, turns out previous library was used incorrectly, this one has simpler API, produces different split points Jarek Kowalski 2017-11-14 22:45:15 -08:00
  • 15602e98f5 output version and build info Jarek Kowalski 2017-11-12 22:43:36 -08:00
  • 40a367f79e added 'block rm' subcommand Jarek Kowalski 2017-11-12 22:43:21 -08:00
  • 585b6ddcc1 additional test for race involving blockmanager.delete Jarek Kowalski 2017-11-12 20:43:33 -08:00
  • c0d71714e1 fixed a case where we would write a pack entry not corresponding to any block Jarek Kowalski 2017-11-12 18:52:32 -08:00
  • 50398692fe added support for block manager delete Jarek Kowalski 2017-11-11 10:06:27 -08:00
  • 5b287815ad add snapshot manifest to manifest.Manager at the end of upload Jarek Kowalski 2017-11-10 20:18:57 -08:00
  • 9c5c51dda7 deployment work Jarek Kowalski 2017-11-08 22:23:46 -08:00
  • 8982c985c9 travis creds Jarek Kowalski 2017-11-08 22:09:06 -08:00
  • 4967f6d406 fixed releasing Jarek Kowalski 2017-11-08 21:10:26 -08:00
  • 032e3cb737 merged 'format' and 'repo' blocks into 'kopia.repository' Jarek Kowalski 2017-11-08 20:09:06 -08:00
  • 2f25d8b9e8 initial manifest manager, not used yet Jarek Kowalski 2017-11-08 18:58:41 -08:00
  • 48a88eee09 removed dead code from metadata manage Jarek Kowalski 2017-11-04 09:36:00 -07:00
  • 113b1f55c7 cleanup Jarek Kowalski 2017-10-29 19:38:07 -07:00
  • 7c41f45e9f extracted block formatting options into a struct under 'block' Jarek Kowalski 2017-10-29 13:00:24 -07:00
  • 49a806154a code reorg Jarek Kowalski 2017-10-29 12:03:49 -07:00
  • 677f0535ed split repo connect/initialize/open into separate files Jarek Kowalski 2017-10-29 11:48:45 -07:00
  • e98a3b3325 tweaks Jarek Kowalski 2017-10-29 11:45:53 -07:00
  • f70d2af03d cleanup Jarek Kowalski 2017-10-29 11:42:11 -07:00
  • 0f6c569435 refactored repository status Jarek Kowalski 2017-10-29 10:35:06 -07:00
  • b6575a97ba moved metadata manager to 'metadata' package and refactored initialization sequence Jarek Kowalski 2017-10-29 10:29:22 -07:00
  • 769c3739bd refactored repository config out of metadata manager Jarek Kowalski 2017-10-29 08:44:53 -07:00
  • ac94dd1705 refactored repo.Connect to avoid calling connect() as most of what it does is not actually needed Jarek Kowalski 2017-10-28 23:40:39 -07:00
  • 3498f78d91 dead code cleanup Jarek Kowalski 2017-10-28 22:50:11 -07:00
  • e76f1f8a72 refactored various show command into one Jarek Kowalski 2017-10-24 21:59:40 -07:00
  • a48d54f00b added 'block show' similar to 'object show', but operating on virtual blocks Jarek Kowalski 2017-10-24 21:15:22 -07:00
  • b4e6a70e09 renamed 'repo optimize' to 'block reindex', added repacking method and unit tests Jarek Kowalski 2017-10-24 21:05:59 -07:00
  • 2aa44eb454 added listing group blocks and summary to 'block list' Jarek Kowalski 2017-10-17 19:24:16 -07:00
  • 42a560fbab block.Manager: Jarek Kowalski 2017-10-16 18:17:45 -07:00
  • 3f20c83e75 avoid storing auth.KeyMaster in metadata.Manager Jarek Kowalski 2017-10-15 09:09:10 -07:00
  • f7a1fb37b0 added auth.KeyManager Jarek Kowalski 2017-10-15 09:00:53 -07:00
  • 44f8940a8f removed support for object ID prefixes Jarek Kowalski 2017-10-14 15:59:23 -07:00
  • a45f557c4b changed object.ManagerOptions into a struct Jarek Kowalski 2017-10-14 15:49:32 -07:00
  • 59bb3ef943 additional test for block manager Jarek Kowalski 2017-10-14 15:43:42 -07:00
  • a715522c34 improved block stats output Jarek Kowalski 2017-10-14 11:19:56 -07:00
  • b5d0f2d120 more renames Jarek Kowalski 2017-10-11 05:47:05 -07:00
  • 0e48800f1a removed ObjectIDSection support, no longer needed because of packs Jarek Kowalski 2017-10-10 22:15:59 -07:00
  • c3f31d744f removed stuttering Jarek Kowalski 2017-10-10 22:09:53 -07:00
  • 31e5bce1cb moved object management to kopia/kopia/object Jarek Kowalski 2017-10-10 22:04:51 -07:00
  • f33ae14573 renamed 'blob.Storage' to 'storage.Storage' Jarek Kowalski 2017-10-09 07:06:20 -07:00
  • cc3e590136 moved block manager and formatter to a separate package Jarek Kowalski 2017-10-09 06:42:30 -07:00
  • a6a37b709a added pack info output to 'block list' Jarek Kowalski 2017-10-08 20:49:25 -07:00
  • ce5d9b2247 output pack group when listing blocks Jarek Kowalski 2017-10-08 17:40:37 -07:00
  • 444b6306ca refactored object manager to use block manager, exposed on repository added CLI commands to manipulate blocks Jarek Kowalski 2017-10-07 15:06:15 -07:00
  • 06efbfbe7a additional test case Jarek Kowalski 2017-10-07 07:24:52 -07:00
  • ff0ac7b8f1 refactored block manager added concurrency and compaction test for block manager Jarek Kowalski 2017-10-06 22:33:06 -07:00
  • c108dbf706 fixed degenerate case of saving index with zero items, added timing info Jarek Kowalski 2017-10-06 19:22:58 -07:00
  • a7818a9a4b fixed handling of zero-byte packs and block manager index flushing order Jarek Kowalski 2017-10-06 06:40:43 -07:00
  • 7071d0b01b refactored block manager, added unit tests Jarek Kowalski 2017-10-05 22:12:02 -07:00
  • 300008c98d renamed pack manager to block manager moved block reading and writing to blockManager Jarek Kowalski 2017-10-03 22:02:23 -07:00
  • 10b9b4b451 fixed repo optimize to remove unused logical blocks Jarek Kowalski 2017-10-02 22:28:00 -07:00
  • a412e45f1f cleanup: bump worker count to 32 Jarek Kowalski 2017-10-01 19:15:32 -07:00
  • 45381eff48 cleanup Jarek Kowalski 2017-10-01 19:15:05 -07:00
  • 903386fb33 removed block size cache, relying on pack manager data instead Jarek Kowalski 2017-10-01 16:05:32 -07:00
  • f96dfe471e support for array pack indexes instead of maps Jarek Kowalski 2017-09-30 17:59:10 -07:00
  • 79440ab969 fixed locking of pack manager, changed how it writes pack and index objects to avoid using object writer, which prevents certain cycles Jarek Kowalski 2017-09-24 21:35:19 -07:00
  • b2d9e07b75 fix 'object list' output. Jarek Kowalski 2017-09-24 12:37:42 -07:00
  • 1366f967f4 pack manager: Jarek Kowalski 2017-09-23 11:49:52 -07:00
  • 1e0ef911fa added 'object list' subcommand Jarek Kowalski 2017-09-23 11:49:04 -07:00
  • 4b77119736 made packing the default, replaced FinishPacking with Flush() Jarek Kowalski 2017-09-23 10:55:57 -07:00
  • 1b548b1ad5 store unpacked blocks in a pack index Jarek Kowalski 2017-09-22 23:01:09 -07:00
  • 4259227eb8 additional diagnostics output Jarek Kowalski 2017-09-18 22:29:24 -07:00
  • 89ddf98564 added 'object verify' subcommand to verify that the given object (file or directory) has all required blocks present removed GetStorageBlock(), replaced with Verify() that also returns storage blocks added blockSizeCache.put() so that object writer can update the cache Jarek Kowalski 2017-09-16 10:47:53 -07:00
  • 32c603362c removed support for inline object IDs now that packs are supported everywhere Jarek Kowalski 2017-09-16 09:27:32 -07:00
  • b5a7d05adf refactored Repository API Jarek Kowalski 2017-09-15 18:35:27 -07:00
  • 91e43b6842 changed how pack files get split by using the same splitter mechanism (static or dynamic) in use for regular objects Jarek Kowalski 2017-09-14 19:17:19 -07:00
  • 0d3dce175c small compaction fix Jarek Kowalski 2017-09-12 06:27:32 -07:00
  • e07c134297 typo Jarek Kowalski 2017-09-11 21:13:43 -07:00
  • 3155aefa59 changed how pack indexes are flushed to avoid prematurely flushing directory packs to make bigger objects Jarek Kowalski 2017-09-11 21:10:29 -07:00
  • 29e0153eff fixed index data corruption during optimize Jarek Kowalski 2017-09-11 20:53:52 -07:00
  • 4010f35a30 fixed deadlock caused by attempt to write asynchronous pack data Jarek Kowalski 2017-09-10 23:08:27 -07:00
  • 207dff1b9f also close current pack when time-based flushing happens Jarek Kowalski 2017-09-10 12:22:28 -07:00
  • 4cfa1d5706 added transparent gunzip to 'obj show' Jarek Kowalski 2017-09-10 12:01:58 -07:00
  • 405ba139ce periodically flush pack indexes every 10 minutes Jarek Kowalski 2017-09-10 11:35:16 -07:00
  • bb64238eba added support for pack index compaction (not the pack file compaction, which is not implemented yet). it is combining multiple pack index objects P* into one large P which loads more quickly Jarek Kowalski 2017-09-09 09:30:06 -07:00
  • 6ef7005396 wait for Ctrl-C when mounting Jarek Kowalski 2017-09-06 18:48:23 -07:00
  • c72d493634 cleanup Jarek Kowalski 2017-09-07 20:01:59 -07:00
  • a7e9f995de removed dependency between pack manager and MetadataManager - all pack indexs are stored as objects with prefix P Jarek Kowalski 2017-09-07 19:58:38 -07:00
  • 976f3a8819 when saving pack index, also save an object with "P" prefix. Jarek 2017-09-06 19:16:54 -07:00
  • fa80f40185 added support for mounting snapshots as drive letters and opening them in explorer on Windows. Jarek 2017-09-05 19:01:51 -07:00
  • 82657f957d extracted timeFormat constant for printing dates in CLI Jarek Kowalski 2017-09-05 18:21:17 -07:00