Jarek Kowalski
06b4bec246
added 'storage sweep' subcommand which removes unreferenced storage blocks
2018-02-10 13:07:10 -08:00
Jarek Kowalski
4971671f3d
added pretty printing of block indexes
2018-02-10 13:07:10 -08:00
Jarek Kowalski
4564bc704f
breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks
...
align pack blocks to 4096 bytes and insert random preamble
2018-02-10 13:07:10 -08:00
Jarek Kowalski
7e16471a21
added --flat option to filesystem and webdav providers
2018-02-10 13:06:34 -08:00
Jarek Kowalski
df35ebad73
added low-level storage viewing subcommands
2018-02-03 22:11:53 -08:00
Jarek Kowalski
7833626c4a
added error handling when unable to load block index
2018-01-24 11:44:26 +00:00
Jarek Kowalski
d7e1276a48
cleaned up storage.Storage API and documentation
2018-01-10 19:13:09 -08:00
Jarek Kowalski
090d97ba78
cli: reorganized all repo commands (connect/create/status) to top level
2018-01-10 19:13:09 -08:00
Jarek Kowalski
61abc8eee2
added support for S3 storage provider
2018-01-10 19:13:09 -08:00
Jarek Kowalski
b65a5c5689
added scrubber to remove passwords from 'repo status' output
2018-01-10 19:13:08 -08:00
Jarek Kowalski
bf4c0e694d
refactored CLI to use individual subcommands to connect to individual storage types, each with specialized flags and validation
2018-01-10 19:13:08 -08:00
Jarek Kowalski
7a97f22df4
switched to OS-specific locations for config and cache
2018-01-06 11:58:46 -08:00
Jarek Kowalski
f9f2c54993
added command to reconfigure caching on a repository that's already connected
2018-01-04 17:44:58 -08:00
Jarek Kowalski
7af93182a4
changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs
2018-01-02 19:51:03 -08:00
Jarek Kowalski
10ca234633
switched to using zerolog for logging, by default log level=info, can be set to debug on commandline
2018-01-01 19:12:35 -08:00
Jarek Kowalski
98070064c8
index block list caching optimization - only load last 2 full compactions when scanning for active index blocks, this prevents the scan time from growing indefinitely
2018-01-01 17:11:41 -08:00
Jarek Kowalski
c288daef04
eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead
2017-11-27 19:15:02 -08:00
Jarek Kowalski
a644f2caa6
added 'manifest show' subcommand
2017-11-27 18:49:46 -08:00
Jarek Kowalski
30c11dc926
refactored block manager to support on-disk caching
2017-11-27 18:07:16 -08:00
Jarek Kowalski
a700508048
changed API for manifest manager, added filtering, sorting and better output to 'manifest ls'
2017-11-27 18:06:28 -08:00
Jarek Kowalski
eebb04c0cb
added 'manifest ls' subcommand
2017-11-27 18:05:44 -08:00
Jarek Kowalski
6e83ad0b9b
moved policies to separate package
...
refactored policy management into policy.Manager that uses manifest.Manager internally
2017-11-27 18:05:32 -08:00
Jarek Kowalski
d3b854ef8e
optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything
2017-11-23 15:55:56 -08:00
Jarek Kowalski
40a367f79e
added 'block rm' subcommand
2017-11-12 22:43:21 -08:00
Jarek Kowalski
0f6c569435
refactored repository status
2017-10-29 10:35:06 -07:00
Jarek Kowalski
b6575a97ba
moved metadata manager to 'metadata' package and refactored initialization sequence
2017-10-29 10:29:22 -07:00
Jarek Kowalski
3498f78d91
dead code cleanup
2017-10-28 22:50:11 -07:00
Jarek Kowalski
e76f1f8a72
refactored various show command into one
2017-10-24 21:59:40 -07:00
Jarek Kowalski
a48d54f00b
added 'block show' similar to 'object show', but operating on virtual blocks
2017-10-24 21:15:22 -07:00
Jarek Kowalski
b4e6a70e09
renamed 'repo optimize' to 'block reindex', added repacking method and unit tests
2017-10-24 21:05:59 -07:00
Jarek Kowalski
2aa44eb454
added listing group blocks and summary to 'block list'
2017-10-17 19:24:16 -07:00
Jarek Kowalski
a45f557c4b
changed object.ManagerOptions into a struct
2017-10-14 16:38:27 -07:00
Jarek Kowalski
a715522c34
improved block stats output
2017-10-14 16:38:27 -07:00
Jarek Kowalski
b5d0f2d120
more renames
2017-10-14 16:38:27 -07:00
Jarek Kowalski
0e48800f1a
removed ObjectIDSection support, no longer needed because of packs
2017-10-14 16:38:22 -07:00
Jarek Kowalski
c3f31d744f
removed stuttering
2017-10-14 16:38:22 -07:00
Jarek Kowalski
31e5bce1cb
moved object management to kopia/kopia/object
2017-10-14 16:38:22 -07:00
Jarek Kowalski
f33ae14573
renamed 'blob.Storage' to 'storage.Storage'
2017-10-14 16:38:09 -07:00
Jarek Kowalski
cc3e590136
moved block manager and formatter to a separate package
2017-10-09 06:42:30 -07:00
Jarek Kowalski
a6a37b709a
added pack info output to 'block list'
2017-10-08 20:49:25 -07:00
Jarek Kowalski
ce5d9b2247
output pack group when listing blocks
2017-10-08 17:40:37 -07:00
Jarek Kowalski
444b6306ca
refactored object manager to use block manager, exposed on repository
...
added CLI commands to manipulate blocks
2017-10-08 12:44:53 -07:00
Jarek Kowalski
10b9b4b451
fixed repo optimize to remove unused logical blocks
2017-10-02 22:28:00 -07:00
Jarek Kowalski
a412e45f1f
cleanup: bump worker count to 32
2017-10-01 19:15:32 -07:00
Jarek Kowalski
f96dfe471e
support for array pack indexes instead of maps
2017-10-01 16:07:55 -07:00
Jarek Kowalski
b2d9e07b75
fix 'object list' output.
2017-09-24 12:38:21 -07:00
Jarek Kowalski
1366f967f4
pack manager:
...
don't register the pack object itself with the pack
added protection from accidental deletion of the block that was just compacted
2017-09-24 12:35:12 -07:00
Jarek Kowalski
1e0ef911fa
added 'object list' subcommand
2017-09-23 11:49:04 -07:00
Jarek Kowalski
89ddf98564
added 'object verify' subcommand to verify that the given object (file or directory) has all required blocks present
...
removed GetStorageBlock(), replaced with Verify() that also returns storage blocks
added blockSizeCache.put() so that object writer can update the cache
2017-09-16 16:52:25 -07:00
Jarek Kowalski
32c603362c
removed support for inline object IDs now that packs are supported everywhere
2017-09-16 09:27:32 -07:00
