Jarek Kowalski
6d2d663cac
moved CLI storage support to cli/
2018-08-30 22:08:55 -07:00
Jarek Kowalski
91066f2469
reorganized low-level repository packages by moving them all under kopia/kopia/repo/
2018-08-30 22:01:05 -07:00
Jarek Kowalski
6484a55d7c
removed policy.Manager and snapshot.Manager and switched to package-level function interface instead
2018-08-30 06:47:58 -07:00
Jarek Kowalski
a44c9554c0
refactored upload to separate package
2018-08-30 06:21:40 -07:00
Jarek Kowalski
3ef4593b03
refactored policy management into separate policy/ package
2018-08-29 23:08:40 -07:00
Jarek Kowalski
9060e7d310
moved repository VFS nodes into fs/repofs/ package
2018-08-29 22:56:23 -07:00
Jarek Kowalski
0f20965143
added 'blockindex recover' tests
2018-08-25 16:30:11 -07:00
Jarek Kowalski
5dd755fcec
added support for index recovery from pack files, work in progress
2018-08-25 16:29:32 -07:00
Jarek Kowalski
6587ade1f9
cli: added 'policy edit' command which launches an editor and allows user to edit policy JSON directly
2018-08-20 19:07:44 -07:00
Jarek Kowalski
864b763cc0
progress: output the # of files in each directory
2018-08-19 11:48:55 -10:00
Jarek Kowalski
1ff003c820
added internal lib for gitignore-style pattern matching
...
added 'ignorefs' which transparently hides all files according to .kopiaignore, which is very similar to .gitignore
2018-08-19 11:48:55 -10:00
Jarek Kowalski
5d73d18347
removed Parent from fs.Entry
2018-08-19 11:32:17 -10:00
Jarek Kowalski
9a3a2da5fa
fixed stdout output of 'expire'
2018-07-27 18:28:58 -07:00
Jarek Kowalski
b2b34c1dea
reacted to a change in gometalinter that failed the build
2018-07-10 06:19:22 -07:00
Jarek Kowalski
c57531b751
switched Go logging library to github.com/op/go-logging which lets us override log levels per module
2018-07-09 21:28:12 -07:00
Jarek Kowalski
00db1abb35
added --html option to 'kopia server' which allows colocating HTML apps on the same port
2018-07-01 13:16:58 -07:00
Jarek Kowalski
94965997e8
switched storage logging to be on by default
2018-06-23 13:50:11 -07:00
Jarek Kowalski
ba6419339a
refactored storage.ListBlocks() to be callback based which simplifies usage significantly
2018-06-23 13:47:43 -07:00
Jarek Kowalski
83036f1a0d
upload progress
2018-06-23 13:10:54 -07:00
Jarek Kowalski
91c51f4807
refactored 'block gc' subcommand
2018-06-23 13:05:40 -07:00
Jarek Kowalski
f4424bd79c
parallelized 'block rewrite'
2018-06-23 12:30:59 -07:00
Jarek Kowalski
1defc18300
refactored command_snapshot_expire
2018-06-23 12:30:45 -07:00
Jarek Kowalski
2f1cf8129b
added skeleton of CLI actions for manipulating server sources
2018-06-18 18:02:36 -07:00
Jarek Kowalski
4faf3cd9d0
finalized CLI to manipulate policies
2018-06-16 12:17:58 -07:00
Jarek Kowalski
22b0766570
API server, work in progress
2018-06-10 20:59:49 -07:00
Jarek Kowalski
b105cca96c
added 'kopia server' which runs simple daemon with simple HTTP API server, which will be used to build a web UI
2018-06-10 17:24:42 -07:00
Jarek Kowalski
846a46b879
changed how caching is controlled in block manager by using context instead of options
2018-06-10 10:20:52 -07:00
Jarek Kowalski
6838d820c8
revamped object verification to be parallelized, added option to randomly verify a percentage of object contents
2018-06-09 22:15:36 -07:00
Jarek Kowalski
e5e5577cfd
fixed linter warnings
2018-06-09 14:06:43 -07:00
Jarek Kowalski
fc618b0c66
changed block.PhysicalBlockID to a string
2018-06-07 18:42:22 -07:00
Jarek Kowalski
6ee5c81e21
renamed 'pack block ID' to 'pack file' everywhere
2018-06-07 07:52:17 -07:00
Jarek Kowalski
f869996a1c
aligned 'ls' output
2018-06-06 21:38:32 -07:00
Jarek Kowalski
9ea74b2862
when reporting stats to GA do it asynchronously at start and do not block for more than 1s on exit
2018-06-06 21:38:08 -07:00
Jarek Kowalski
7738fa536f
revamped 'object verify' to support verifying one or more sources easily
2018-06-06 21:18:44 -07:00
Jarek Kowalski
ebd8b0a943
CLI improvements
2018-06-04 22:55:57 -07:00
Jarek Kowalski
416475a092
added RewriteBlock() support which can rewrite either deleted or non-deleted block to a new pack
...
Added 'block gc' and 'block rewrite' command-line subcommands.
2018-06-04 22:09:10 -07:00
Jarek Kowalski
02c5e90671
added new cli flags to 'storage list'
2018-06-02 10:46:59 -07:00
Jarek Kowalski
5ba913380e
added support for soft-delete that preserves original pack ID/offset/length.
...
removed legacy soft-delete support without this information
2018-05-30 21:22:07 -07:00
Jarek Kowalski
53db414ff7
Added simple analytics mechanism based on Google Analytics for tracking features usage and latency.
...
Controlled on command line via --analytics-consent (defaults to asking user)
2018-05-30 21:22:07 -07:00
Jarek Kowalski
80a3bdfd4a
fixed race condition when multiple kopia instances are downloading indexes in parallel
2018-05-20 10:45:13 -07:00
Jarek Kowalski
e36d02c59b
emit per-dir incomplete reason
...
store entire rootEntry on snapshot nmanifest
format 'snapshot list' consistently for both root and nested items
2018-05-19 22:00:14 -07:00
Jarek Kowalski
e6c8144b93
output pack block id in 'blockindex show active'
2018-05-17 23:26:43 -07:00
Jarek Kowalski
96d27c2293
improved block list and index format
2018-05-17 22:41:54 -07:00
Jarek Kowalski
ec779b14c7
refactored and cleaned up policy management, added retention tags to snapshot list
2018-05-17 20:31:37 -07:00
Jarek Kowalski
c907580aaf
when running 'snapshot create --all' ignore individual source error failures
2018-05-15 20:54:45 -07:00
Jarek Kowalski
d6477377f9
changed how index compaction works:
...
instead of having time-based naming, block manager will perform occasional compaction at startup time and delete unwanted blocks
The protocol is safe from concurrency standpoint and provides eventual consistency.
We only delete blocks after creating compacted blocks.
We retry loading index until (any) consistent index is fetched (possibly from cache) and all underlying blocks are also available, not necessarily the latest ones.
TODO - we need to periodically snapshot block index contents, so that if we have a bug somewhere in compaction code, we have a way of restoring working indexes.
2018-05-15 20:10:50 -07:00
Jarek Kowalski
7e6154f6f3
fixed progress bar
2018-05-13 21:55:00 -07:00
Jarek Kowalski
12c28e537d
simplified ObjectID code and changed it to a string
2018-05-09 22:22:37 -07:00
Jarek Kowalski
62ce401cd6
exposed block.Manager.ListBlocks() which only returns block IDs instead of full info
2018-05-05 16:16:48 -07:00
Jarek Kowalski
3a02ef24e0
retire block.ContentID and replace with string
2018-05-05 16:16:48 -07:00
Jarek Kowalski