Jarek Kowalski
42a560fbab
block.Manager:
...
Maintaining per-group block index
Added block.Manager.ListGroupBlocks() API + tests
2017-10-16 19:19:51 -07:00
Jarek Kowalski
3f20c83e75
avoid storing auth.KeyMaster in metadata.Manager
2017-10-15 09:09:10 -07:00
Jarek Kowalski
f7a1fb37b0
added auth.KeyManager
2017-10-15 09:00:53 -07:00
Jarek Kowalski
44f8940a8f
removed support for object ID prefixes
2017-10-14 16:38:27 -07:00
Jarek Kowalski
a45f557c4b
changed object.ManagerOptions into a struct
2017-10-14 16:38:27 -07:00
Jarek Kowalski
59bb3ef943
additional test for block manager
2017-10-14 16:38:27 -07:00
Jarek Kowalski
a715522c34
improved block stats output
2017-10-14 16:38:27 -07:00
Jarek Kowalski
b5d0f2d120
more renames
2017-10-14 16:38:27 -07:00
Jarek Kowalski
0e48800f1a
removed ObjectIDSection support, no longer needed because of packs
2017-10-14 16:38:22 -07:00
Jarek Kowalski
c3f31d744f
removed stuttering
2017-10-14 16:38:22 -07:00
Jarek Kowalski
31e5bce1cb
moved object management to kopia/kopia/object
2017-10-14 16:38:22 -07:00
Jarek Kowalski
f33ae14573
renamed 'blob.Storage' to 'storage.Storage'
2017-10-14 16:38:09 -07:00
Jarek Kowalski
cc3e590136
moved block manager and formatter to a separate package
2017-10-09 06:42:30 -07:00
Jarek Kowalski
a6a37b709a
added pack info output to 'block list'
2017-10-08 20:49:25 -07:00
Jarek Kowalski
ce5d9b2247
output pack group when listing blocks
2017-10-08 17:40:37 -07:00
Jarek Kowalski
444b6306ca
refactored object manager to use block manager, exposed on repository
...
added CLI commands to manipulate blocks
2017-10-08 12:44:53 -07:00
Jarek Kowalski
06efbfbe7a
additional test case
2017-10-07 07:24:52 -07:00
Jarek Kowalski
ff0ac7b8f1
refactored block manager
...
added concurrency and compaction test for block manager
2017-10-07 07:22:04 -07:00
Jarek Kowalski
c108dbf706
fixed degenerate case of saving index with zero items, added timing info
2017-10-06 19:22:58 -07:00
Jarek Kowalski
a7818a9a4b
fixed handling of zero-byte packs and block manager index flushing order
2017-10-06 06:40:43 -07:00
Jarek Kowalski
7071d0b01b
refactored block manager, added unit tests
2017-10-05 22:12:02 -07:00
Jarek Kowalski
300008c98d
renamed pack manager to block manager
...
moved block reading and writing to blockManager
2017-10-03 23:18:16 -07:00
Jarek Kowalski
10b9b4b451
fixed repo optimize to remove unused logical blocks
2017-10-02 22:28:00 -07:00
Jarek Kowalski
a412e45f1f
cleanup: bump worker count to 32
2017-10-01 19:15:32 -07:00
Jarek Kowalski
45381eff48
cleanup
2017-10-01 19:15:05 -07:00
Jarek Kowalski
903386fb33
removed block size cache, relying on pack manager data instead
2017-10-01 16:22:26 -07:00
Jarek Kowalski
f96dfe471e
support for array pack indexes instead of maps
2017-10-01 16:07:55 -07:00
Jarek Kowalski
79440ab969
fixed locking of pack manager, changed how it writes pack and index objects to avoid using object writer, which prevents certain cycles
2017-09-24 21:35:19 -07:00
Jarek Kowalski
b2d9e07b75
fix 'object list' output.
2017-09-24 12:38:21 -07:00
Jarek Kowalski
1366f967f4
pack manager:
...
don't register the pack object itself with the pack
added protection from accidental deletion of the block that was just compacted
2017-09-24 12:35:12 -07:00
Jarek Kowalski
1e0ef911fa
added 'object list' subcommand
2017-09-23 11:49:04 -07:00
Jarek Kowalski
4b77119736
made packing the default, replaced FinishPacking with Flush()
2017-09-23 10:55:57 -07:00
Jarek Kowalski
1b548b1ad5
store unpacked blocks in a pack index
2017-09-22 23:01:09 -07:00
Jarek Kowalski
4259227eb8
additional diagnostics output
2017-09-18 22:29:24 -07:00
Jarek Kowalski
89ddf98564
added 'object verify' subcommand to verify that the given object (file or directory) has all required blocks present
...
removed GetStorageBlock(), replaced with Verify() that also returns storage blocks
added blockSizeCache.put() so that object writer can update the cache
2017-09-16 16:52:25 -07:00
Jarek Kowalski
32c603362c
removed support for inline object IDs now that packs are supported everywhere
2017-09-16 09:27:32 -07:00
Jarek Kowalski
b5a7d05adf
refactored Repository API
2017-09-15 18:35:27 -07:00
Jarek Kowalski
91e43b6842
changed how pack files get split by using the same splitter mechanism (static or dynamic) in use for regular objects
2017-09-14 19:17:19 -07:00
Jarek Kowalski
0d3dce175c
small compaction fix
2017-09-12 06:27:32 -07:00
Jarek Kowalski
e07c134297
typo
2017-09-11 21:13:43 -07:00
Jarek Kowalski
3155aefa59
changed how pack indexes are flushed to avoid prematurely flushing directory packs to make bigger objects
2017-09-11 21:10:29 -07:00
Jarek Kowalski
29e0153eff
fixed index data corruption during optimize
2017-09-11 20:53:52 -07:00
Jarek Kowalski
4010f35a30
fixed deadlock caused by attempt to write asynchronous pack data
2017-09-10 23:08:27 -07:00
Jarek Kowalski
207dff1b9f
also close current pack when time-based flushing happens
2017-09-10 12:22:28 -07:00
Jarek Kowalski
4cfa1d5706
added transparent gunzip to 'obj show'
2017-09-10 12:01:58 -07:00
Jarek Kowalski
405ba139ce
periodically flush pack indexes every 10 minutes
2017-09-10 11:35:32 -07:00
Jarek Kowalski
bb64238eba
added support for pack index compaction (not the pack file compaction, which is not implemented yet). it is combining multiple pack index objects P* into one large P which loads more quickly
2017-09-09 11:07:59 -07:00
Jarek Kowalski
6ef7005396
wait for Ctrl-C when mounting
2017-09-07 20:05:58 -07:00
Jarek Kowalski
c72d493634
cleanup
2017-09-07 20:01:59 -07:00
Jarek Kowalski
a7e9f995de
removed dependency between pack manager and MetadataManager - all pack indexs are stored as objects with prefix P
2017-09-07 19:58:38 -07:00
