Commit Graph

3660 Commits

Author SHA1 Message Date
Jarek Kowalski
00db1abb35 added --html option to 'kopia server' which allows colocating HTML apps on the same port 2018-07-01 13:16:58 -07:00
Jarek Kowalski
75da8f3f31 removed dead code 2018-06-24 15:04:58 -07:00
Jarek Kowalski
cbcdfa2eb6 switched coverage to native go test 2018-06-23 16:32:47 -07:00
Jarek Kowalski
94965997e8 switched storage logging to be on by default 2018-06-23 13:50:11 -07:00
Jarek Kowalski
ba6419339a refactored storage.ListBlocks() to be callback based which simplifies usage significantly 2018-06-23 13:47:43 -07:00
Jarek Kowalski
83036f1a0d upload progress 2018-06-23 13:10:54 -07:00
Jarek Kowalski
91c51f4807 refactored 'block gc' subcommand 2018-06-23 13:05:40 -07:00
Jarek Kowalski
f4424bd79c parallelized 'block rewrite' 2018-06-23 12:30:59 -07:00
Jarek Kowalski
1defc18300 refactored command_snapshot_expire 2018-06-23 12:30:45 -07:00
Jarek Kowalski
c7d898a674 fixed block manager lock which triggered asserts during parallel read/write 2018-06-23 12:30:19 -07:00
Jarek Kowalski
173c5613c1 fixed aliasing of read buffers in block manager 2018-06-18 19:16:13 -07:00
Jarek Kowalski
85b4bb7cd9 better fix for aliasing issue 2018-06-18 18:23:37 -07:00
Jarek Kowalski
23318a461d fixed rare data corruption bug due to buffer aliasing 2018-06-18 18:02:36 -07:00
Jarek Kowalski
2f1cf8129b added skeleton of CLI actions for manipulating server sources 2018-06-18 18:02:36 -07:00
Jarek Kowalski
4faf3cd9d0 finalized CLI to manipulate policies 2018-06-16 12:17:58 -07:00
Jarek Kowalski
70ecebb769 added HTTP API to list policies 2018-06-16 08:46:56 -07:00
Jarek Kowalski
a6f2e8c3f3 added option to disable block manager flush to ensure that a sequence of writes/deletes is atomically observable 2018-06-14 17:11:00 -07:00
Jarek Kowalski
5333a9cfc1 reorganized dynamic splitter logic to be slightly faster and ignore hash value of zero, which introduced blocks smaller than expected on average 2018-06-13 22:56:17 -07:00
Jarek Kowalski
6b79ab79d0 fixed --no-list-caching 2018-06-13 22:37:45 -07:00
Jarek Kowalski
d3cf01fc45 present legacy object IDs without the 'D' 2018-06-13 22:35:46 -07:00
Jarek Kowalski
7dea2359c2 removed 'D' prefix from direct objects 2018-06-12 20:39:53 -07:00
Jarek Kowalski
e21a5ce650 refactored committed block index to separate cache from the lookup 2018-06-12 20:22:37 -07:00
Jarek Kowalski
df3129682c Added repo.Repository.Refresh{,Periodically} 2018-06-12 18:03:25 -07:00
Jarek Kowalski
2be6041d96 rewrote manifest manager to use distinct committed and pending layers 2018-06-10 21:35:59 -07:00
Jarek Kowalski
22b0766570 API server, work in progress 2018-06-10 20:59:49 -07:00
Jarek Kowalski
4313d5f6ab added block.Manager.Refresh method 2018-06-10 20:59:49 -07:00
Jarek Kowalski
b105cca96c added 'kopia server' which runs simple daemon with simple HTTP API server, which will be used to build a web UI 2018-06-10 17:24:42 -07:00
Jarek Kowalski
e9c11ea106 refactored block cache and list cache 2018-06-10 16:09:47 -07:00
Jarek Kowalski
846a46b879 changed how caching is controlled in block manager by using context instead of options 2018-06-10 10:20:52 -07:00
Jarek Kowalski
9a17179615 reorganized local cache to have 256 shards instead of tens of thousands, this improves scan speed and startup time with lots of cached items 2018-06-10 08:15:07 -07:00
Jarek Kowalski
6838d820c8 revamped object verification to be parallelized, added option to randomly verify a percentage of object contents 2018-06-09 22:15:36 -07:00
Jarek Kowalski
19dcda1f47 removed lock while loading a block that made block manager unnecessarily single-threaded 2018-06-09 18:36:59 -07:00
Jarek Kowalski
e5e5577cfd fixed linter warnings 2018-06-09 14:06:43 -07:00
Jarek Kowalski
fc618b0c66 changed block.PhysicalBlockID to a string 2018-06-07 18:42:22 -07:00
Jarek Kowalski
6ee5c81e21 renamed 'pack block ID' to 'pack file' everywhere 2018-06-07 07:52:17 -07:00
Jarek Kowalski
f869996a1c aligned 'ls' output 2018-06-06 21:38:32 -07:00
Jarek Kowalski
9ea74b2862 when reporting stats to GA do it asynchronously at start and do not block for more than 1s on exit 2018-06-06 21:38:08 -07:00
Jarek Kowalski
7738fa536f revamped 'object verify' to support verifying one or more sources easily 2018-06-06 21:18:44 -07:00
Jarek Kowalski
8d751cb7bf added 'k' prefix to all directory blocks and 'h' prefix to all hash-cache blocks 2018-06-06 21:15:52 -07:00
Jarek Kowalski
985980925f removed support for v0 block format 2018-06-05 20:17:21 -07:00
Jarek Kowalski
ebd8b0a943 CLI improvements 2018-06-04 22:55:57 -07:00
Jarek Kowalski
416475a092 added RewriteBlock() support which can rewrite either deleted or non-deleted block to a new pack
Added 'block gc' and 'block rewrite' command-line subcommands.
2018-06-04 22:09:10 -07:00
Jarek Kowalski
438fa4a587 whenever two blocks in index have identical times, prefer the non-deleted one 2018-06-04 22:08:58 -07:00
Jarek Kowalski
f6dcd3c155 remove old unused cached index blocks 2018-06-02 10:51:32 -07:00
Jarek Kowalski
02c5e90671 added new cli flags to 'storage list' 2018-06-02 10:46:59 -07:00
Jarek Kowalski
5ba913380e added support for soft-delete that preserves original pack ID/offset/length.
removed legacy soft-delete support without this information
2018-05-30 21:22:07 -07:00
Jarek Kowalski
53db414ff7 Added simple analytics mechanism based on Google Analytics for tracking features usage and latency.
Controlled on command line via --analytics-consent (defaults to asking user)
2018-05-30 21:22:07 -07:00
Jarek Kowalski
80a3bdfd4a fixed race condition when multiple kopia instances are downloading indexes in parallel 2018-05-20 10:45:13 -07:00
Jarek Kowalski
e36d02c59b emit per-dir incomplete reason
store entire rootEntry on snapshot nmanifest
format 'snapshot list' consistently for both root and nested items
2018-05-19 22:00:14 -07:00
Jarek Kowalski
37566b0a05 output per-directory summary when uploading 2018-05-19 07:31:06 -07:00