Jarek Kowalski
|
e710dcdff3
|
additional integration tests
|
2018-02-18 11:59:32 -08:00 |
|
Jarek Kowalski
|
a92ea34559
|
upgrade go to 1.10, try #2
|
2018-02-18 11:00:19 -08:00 |
|
Jarek Kowalski
|
bd06941b8e
|
upgrade go to 1.10
|
2018-02-18 10:58:24 -08:00 |
|
Jarek Kowalski
|
663fee1ef1
|
rudimentary end-to-end tests to be running as part of travis
|
2018-02-18 10:52:15 -08:00 |
|
Jarek Kowalski
|
986dc02ad9
|
ignore manifest entries marked as deleted
|
2018-02-18 10:39:16 -08:00 |
|
Jarek Kowalski
|
b6ae16bfb6
|
changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs
|
2018-02-16 18:18:47 -08:00 |
|
Jarek Kowalski
|
faa2625a5f
|
revamped CLI help to hide most commands
|
2018-02-15 19:49:51 -08:00 |
|
Jarek Kowalski
|
01ffec69fc
|
fixed output of 'storage list' for long block names
|
2018-02-11 08:12:20 -08:00 |
|
Jarek Kowalski
|
4c5377898f
|
fixed handling of degenerate cases in 'block repack'
|
2018-02-11 08:11:31 -08:00 |
|
Jarek Kowalski
|
a350991762
|
reorganized CLI packages to be close to storage providers
|
2018-02-11 08:00:07 -08:00 |
|
Jarek Kowalski
|
16095ae900
|
removed block prefixes in object manager, changed metadata prefix to lowercase m
|
2018-02-10 19:47:30 -08:00 |
|
Jarek Kowalski
|
06b4bec246
|
added 'storage sweep' subcommand which removes unreferenced storage blocks
|
2018-02-10 13:07:10 -08:00 |
|
Jarek Kowalski
|
4971671f3d
|
added pretty printing of block indexes
|
2018-02-10 13:07:10 -08:00 |
|
Jarek Kowalski
|
4564bc704f
|
breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks
align pack blocks to 4096 bytes and insert random preamble
|
2018-02-10 13:07:10 -08:00 |
|
Jarek Kowalski
|
7e16471a21
|
added --flat option to filesystem and webdav providers
|
2018-02-10 13:06:34 -08:00 |
|
Jarek Kowalski
|
df35ebad73
|
added low-level storage viewing subcommands
|
2018-02-03 22:11:53 -08:00 |
|
Jarek Kowalski
|
14ea44d02d
|
fixed S3 test by recreating bucket and using per-host bucket
|
2018-02-03 17:19:37 -08:00 |
|
Jarek Kowalski
|
7833626c4a
|
added error handling when unable to load block index
|
2018-01-24 11:44:26 +00:00 |
|
Jarek Kowalski
|
d7e1276a48
|
cleaned up storage.Storage API and documentation
|
2018-01-10 19:13:09 -08:00 |
|
Jarek Kowalski
|
090d97ba78
|
cli: reorganized all repo commands (connect/create/status) to top level
|
2018-01-10 19:13:09 -08:00 |
|
Jarek Kowalski
|
61abc8eee2
|
added support for S3 storage provider
|
2018-01-10 19:13:09 -08:00 |
|
Jarek Kowalski
|
b6611d2131
|
removed depedendency on actual providers from 'repo'
|
2018-01-10 19:13:09 -08:00 |
|
Jarek Kowalski
|
b65a5c5689
|
added scrubber to remove passwords from 'repo status' output
|
2018-01-10 19:13:08 -08:00 |
|
Jarek Kowalski
|
bf4c0e694d
|
refactored CLI to use individual subcommands to connect to individual storage types, each with specialized flags and validation
|
2018-01-10 19:13:08 -08:00 |
|
Jarek Kowalski
|
9bf5d4a8c2
|
added colorized logging on Windows, minor tweaks
|
2018-01-06 12:04:39 -08:00 |
|
Jarek Kowalski
|
7a97f22df4
|
switched to OS-specific locations for config and cache
|
2018-01-06 11:58:46 -08:00 |
|
Jarek Kowalski
|
393e7a8d59
|
disabled ANSI escape sequences when logging on windows
|
2018-01-04 17:45:15 -08:00 |
|
Jarek Kowalski
|
f9f2c54993
|
added command to reconfigure caching on a repository that's already connected
|
2018-01-04 17:44:58 -08:00 |
|
Jarek Kowalski
|
7af93182a4
|
changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs
|
2018-01-02 19:51:03 -08:00 |
|
Jarek Kowalski
|
10ca234633
|
switched to using zerolog for logging, by default log level=info, can be set to debug on commandline
|
2018-01-01 19:12:35 -08:00 |
|
Jarek Kowalski
|
ef2ac1b5eb
|
dead code
|
2018-01-01 17:16:51 -08:00 |
|
Jarek Kowalski
|
9bd93ee787
|
stale comment
|
2018-01-01 17:12:56 -08:00 |
|
Jarek Kowalski
|
98070064c8
|
index block list caching optimization - only load last 2 full compactions when scanning for active index blocks, this prevents the scan time from growing indefinitely
|
2018-01-01 17:11:41 -08:00 |
|
Jarek Kowalski
|
6dda6ca5e5
|
prefix each index block name with an inverse timestamp, that decreases over time an can be lexicographically sorted to produce newest blocks first when listing by prefix P
|
2017-12-31 16:16:10 -08:00 |
|
Jarek Kowalski
|
596c2b122a
|
disk cache: remove list after a write
fix close
|
2017-12-31 16:15:58 -08:00 |
|
Jarek Kowalski
|
51c1c77fb8
|
map initialization tweak
|
2017-11-27 19:40:23 -08:00 |
|
Jarek Kowalski
|
7f73ca2e52
|
added missing comments, cleanup
|
2017-11-27 19:34:07 -08:00 |
|
Jarek Kowalski
|
c288daef04
|
eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead
|
2017-11-27 19:15:02 -08:00 |
|
Jarek Kowalski
|
a644f2caa6
|
added 'manifest show' subcommand
|
2017-11-27 18:49:46 -08:00 |
|
Jarek Kowalski
|
30c11dc926
|
refactored block manager to support on-disk caching
|
2017-11-27 18:07:16 -08:00 |
|
Jarek Kowalski
|
a0cc09b673
|
removed caching storage
|
2017-11-27 18:06:28 -08:00 |
|
Jarek Kowalski
|
a700508048
|
changed API for manifest manager, added filtering, sorting and better output to 'manifest ls'
|
2017-11-27 18:06:28 -08:00 |
|
Jarek Kowalski
|
1968e2fa10
|
fix support for encrypted blocks with dash suffix
|
2017-11-27 18:05:44 -08:00 |
|
Jarek Kowalski
|
eebb04c0cb
|
added 'manifest ls' subcommand
|
2017-11-27 18:05:44 -08:00 |
|
Jarek Kowalski
|
6e83ad0b9b
|
moved policies to separate package
refactored policy management into policy.Manager that uses manifest.Manager internally
|
2017-11-27 18:05:32 -08:00 |
|
Jarek Kowalski
|
de02941442
|
plumbed through key time in map storage
|
2017-11-23 16:43:34 -08:00 |
|
Jarek Kowalski
|
d3b854ef8e
|
optimized format for pack index, added compacted block with suffix -zTIMESTAMP where TIMESTAMP is base16-encoded unix nanoseconds of all blocks that this block supersedes, this allows much more efficient loading of blocks without having to delete anything
|
2017-11-23 15:55:56 -08:00 |
|
Jarek Kowalski
|
29ed48f734
|
manifest manager compaction
|
2017-11-23 10:49:06 -08:00 |
|
Jarek Kowalski
|
74195e316e
|
switched buzhash implementation to github.com/silvasur/buzhash, turns out previous library was used incorrectly, this one has simpler API, produces different split points
|
2017-11-14 22:45:15 -08:00 |
|
Jarek Kowalski
|
15602e98f5
|
output version and build info
|
2017-11-12 22:43:36 -08:00 |
|