Jarek Kowalski
14e38b59bf
cleaned up how active block index is loaded
2018-04-14 19:26:58 -07:00
Jarek Kowalski
924e0a4594
renamed index proto types to IndexV1
2018-04-08 18:45:22 -07:00
Jarek Kowalski
d8201229d8
plumbed through ctx in storage.Storage APIs and all uses
2018-04-03 17:39:54 -07:00
Jarek Kowalski
2b49a5df7b
added per-directory summary
2018-03-24 12:22:21 -07:00
Jarek Kowalski
e997e07b0c
added support for summaries to JSON stream
2018-03-24 12:22:21 -07:00
Jarek Kowalski
1c4e0e1e47
removed legacy oid format
2018-03-19 19:31:03 -07:00
Jarek Kowalski
453bab3560
fixed some lint errors, mostly dead code and missing error checks
2018-03-19 12:26:28 -07:00
Jarek Kowalski
4657f16cc8
CLI improvement for automatic logging and log rotation
2018-03-16 22:21:57 -07:00
Jarek Kowalski
f4c9749246
additional logging for hash cache
2018-03-12 16:49:45 -07:00
Jarek Kowalski
0fa823976f
changed upload order to post-order - we first preprocess all subdirectories of a given directory, followed by files
2018-03-12 09:17:27 -07:00
Jarek Kowalski
b6ae16bfb6
changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs
2018-02-16 18:18:47 -08:00
Jarek Kowalski
4564bc704f
breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks
...
align pack blocks to 4096 bytes and insert random preamble
2018-02-10 13:07:10 -08:00
Jarek Kowalski
d7e1276a48
cleaned up storage.Storage API and documentation
2018-01-10 19:13:09 -08:00
Jarek Kowalski
b65a5c5689
added scrubber to remove passwords from 'repo status' output
2018-01-10 19:13:08 -08:00
Jarek Kowalski
7a97f22df4
switched to OS-specific locations for config and cache
2018-01-06 11:58:46 -08:00
Jarek Kowalski
7af93182a4
changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs
2018-01-02 19:51:03 -08:00
Jarek Kowalski
10ca234633
switched to using zerolog for logging, by default log level=info, can be set to debug on commandline
2018-01-01 19:12:35 -08:00
Jarek Kowalski
30c11dc926
refactored block manager to support on-disk caching
2017-11-27 18:07:16 -08:00
Jarek Kowalski
de02941442
plumbed through key time in map storage
2017-11-23 16:43:34 -08:00
Jarek Kowalski
7c41f45e9f
extracted block formatting options into a struct under 'block'
2017-10-29 13:00:24 -07:00
Jarek Kowalski
49a806154a
code reorg
2017-10-29 12:03:49 -07:00
Jarek Kowalski
b6575a97ba
moved metadata manager to 'metadata' package and refactored initialization sequence
2017-10-29 10:29:22 -07:00
Jarek Kowalski
b5d0f2d120
more renames
2017-10-14 16:38:27 -07:00
Jarek Kowalski
c3f31d744f
removed stuttering
2017-10-14 16:38:22 -07:00
Jarek Kowalski
31e5bce1cb
moved object management to kopia/kopia/object
2017-10-14 16:38:22 -07:00
Jarek Kowalski
f33ae14573
renamed 'blob.Storage' to 'storage.Storage'
2017-10-14 16:38:09 -07:00
Jarek Kowalski
cc3e590136
moved block manager and formatter to a separate package
2017-10-09 06:42:30 -07:00
Jarek Kowalski
444b6306ca
refactored object manager to use block manager, exposed on repository
...
added CLI commands to manipulate blocks
2017-10-08 12:44:53 -07:00
Jarek Kowalski
91e43b6842
changed how pack files get split by using the same splitter mechanism (static or dynamic) in use for regular objects
2017-09-14 19:17:19 -07:00
Jarek Kowalski
b08a35e92b
refactored FS cache into reusable wrapper that can be applied on top of any filesystem abstraction
2017-09-04 17:42:50 -07:00
Jarek Kowalski
e8847b65cc
mounting webdav works, changed how FS directory caching is implemented to be based solely on ObjectIDs
2017-09-02 08:29:59 -07:00
Jarek Kowalski
2c6213239a
added cache expiration to fscache.Cache, so that entries based on ObjectID are cached for a very long time and others expire almost immediately.
...
This allows 'kopia mount all' to see newly created snapshots.
2017-08-29 21:47:06 -07:00
Jarek Kowalski
7b766a7681
fixed liter warnings not detected by old linter
2017-08-27 15:16:27 -07:00
Jarek Kowalski
d613d99f7e
beginnings of WebDAV mounting support, not really working yet, need to do have better caching
2017-08-27 15:02:56 -07:00
Jarek Kowalski
ebe00e9405
fixup
2017-08-27 09:00:32 -07:00
Jarek Kowalski
2b887b0b2e
changed fs.Reader to be a Seeker too
2017-08-27 08:58:59 -07:00
Jarek Kowalski
c3064f56b0
added experimental support for WebDAV backends (use http:// or https://)
2017-08-21 22:17:04 -07:00
Jarek Kowalski
484db94e23
removed support for legacy bundles, now that packs are supported everywhere
2017-08-19 18:06:15 -07:00
Jarek Kowalski
f0fcc60374
now that packing is transparent, inline object IDs are not needed, removed support for them and simplified ObjectWriter.Result() to no longer require a parameter
2017-08-19 14:23:09 -07:00
Jarek Kowalski
3e36ce748f
removed conditional PutBlock, which was causing more trouble that it's worth
...
with unconditional rewrites any partial block writes are self-correcting on subsequent updates,
without it they can't complete at all.
2017-08-13 08:54:50 -07:00
Jarek Kowalski
8dda3bf2c5
added option to read portions of blob storage blocks
2017-08-12 23:11:01 -07:00
Jarek Kowalski
94f5420557
re-added support for bundles (read-only) to aid data migration
2017-08-12 14:07:22 -07:00
Jarek Kowalski
ce4d59d8bb
added support for cancelling upload with Ctrl-C which marks the snapshot as incomplete
...
added --include-incomplete to 'backups' command
improved upload API, added support for better cancellation and upload limits
changed hashcache Reader and Writer to use interfaces
2017-08-10 20:27:31 -07:00
Jarek Kowalski
2c86e54c81
removed support for bundles now that packs are looking much better and more flexible
2017-08-10 18:24:22 -07:00
Jarek Kowalski
6821b8def4
Initial support for creating pack objects, which are identified by
...
P<pack-id>.<block-id> and store smaller files in big objects.
For each pack there is a corresponding metadata object that ObjectID to (offset,length) within a pack object.
2017-08-07 14:18:00 +02:00
Jarek Kowalski
2dfcd23ea9
added hashcache database, based on BoltDB which caches results of computing ObjectID for any given os.FileInfo.
2017-08-04 13:19:28 +02:00
Jarek Kowalski
21a1cbd0bb
renamed auth.Options to auth.SecurityOptions
2017-08-01 17:47:28 +02:00
Jarek Kowalski
cfbef13aff
Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
...
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4
removed 'client' package and merged with 'repo', names still not final
2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc
replaced use of Vault with Repository
2017-08-01 08:29:41 +02:00
