Jarek Kowalski
b6ae16bfb6
changed how small files are stored to avoid creating very small data files by embedding them directly in the index packs
2018-02-16 18:18:47 -08:00
Jarek Kowalski
4564bc704f
breaking format change: removed support for groups in block manager, this produces bigger, tightly packed blocks
...
align pack blocks to 4096 bytes and insert random preamble
2018-02-10 13:07:10 -08:00
Jarek Kowalski
d7e1276a48
cleaned up storage.Storage API and documentation
2018-01-10 19:13:09 -08:00
Jarek Kowalski
b65a5c5689
added scrubber to remove passwords from 'repo status' output
2018-01-10 19:13:08 -08:00
Jarek Kowalski
7a97f22df4
switched to OS-specific locations for config and cache
2018-01-06 11:58:46 -08:00
Jarek Kowalski
7af93182a4
changed format of block index to be based on protobuf (using gogo), the deserialization is 10-12x faster than JSON, which matters for a large number of packs
2018-01-02 19:51:03 -08:00
Jarek Kowalski
10ca234633
switched to using zerolog for logging, by default log level=info, can be set to debug on commandline
2018-01-01 19:12:35 -08:00
Jarek Kowalski
30c11dc926
refactored block manager to support on-disk caching
2017-11-27 18:07:16 -08:00
Jarek Kowalski
de02941442
plumbed through key time in map storage
2017-11-23 16:43:34 -08:00
Jarek Kowalski
7c41f45e9f
extracted block formatting options into a struct under 'block'
2017-10-29 13:00:24 -07:00
Jarek Kowalski
49a806154a
code reorg
2017-10-29 12:03:49 -07:00
Jarek Kowalski
b6575a97ba
moved metadata manager to 'metadata' package and refactored initialization sequence
2017-10-29 10:29:22 -07:00
Jarek Kowalski
b5d0f2d120
more renames
2017-10-14 16:38:27 -07:00
Jarek Kowalski
c3f31d744f
removed stuttering
2017-10-14 16:38:22 -07:00
Jarek Kowalski
31e5bce1cb
moved object management to kopia/kopia/object
2017-10-14 16:38:22 -07:00
Jarek Kowalski
f33ae14573
renamed 'blob.Storage' to 'storage.Storage'
2017-10-14 16:38:09 -07:00
Jarek Kowalski
cc3e590136
moved block manager and formatter to a separate package
2017-10-09 06:42:30 -07:00
Jarek Kowalski
444b6306ca
refactored object manager to use block manager, exposed on repository
...
added CLI commands to manipulate blocks
2017-10-08 12:44:53 -07:00
Jarek Kowalski
91e43b6842
changed how pack files get split by using the same splitter mechanism (static or dynamic) in use for regular objects
2017-09-14 19:17:19 -07:00
Jarek Kowalski
b08a35e92b
refactored FS cache into reusable wrapper that can be applied on top of any filesystem abstraction
2017-09-04 17:42:50 -07:00
Jarek Kowalski
e8847b65cc
mounting webdav works, changed how FS directory caching is implemented to be based solely on ObjectIDs
2017-09-02 08:29:59 -07:00
Jarek Kowalski
2c6213239a
added cache expiration to fscache.Cache, so that entries based on ObjectID are cached for a very long time and others expire almost immediately.
...
This allows 'kopia mount all' to see newly created snapshots.
2017-08-29 21:47:06 -07:00
Jarek Kowalski
7b766a7681
fixed liter warnings not detected by old linter
2017-08-27 15:16:27 -07:00
Jarek Kowalski
d613d99f7e
beginnings of WebDAV mounting support, not really working yet, need to do have better caching
2017-08-27 15:02:56 -07:00
Jarek Kowalski
ebe00e9405
fixup
2017-08-27 09:00:32 -07:00
Jarek Kowalski
2b887b0b2e
changed fs.Reader to be a Seeker too
2017-08-27 08:58:59 -07:00
Jarek Kowalski
c3064f56b0
added experimental support for WebDAV backends (use http:// or https://)
2017-08-21 22:17:04 -07:00
Jarek Kowalski
484db94e23
removed support for legacy bundles, now that packs are supported everywhere
2017-08-19 18:06:15 -07:00
Jarek Kowalski
f0fcc60374
now that packing is transparent, inline object IDs are not needed, removed support for them and simplified ObjectWriter.Result() to no longer require a parameter
2017-08-19 14:23:09 -07:00
Jarek Kowalski
3e36ce748f
removed conditional PutBlock, which was causing more trouble that it's worth
...
with unconditional rewrites any partial block writes are self-correcting on subsequent updates,
without it they can't complete at all.
2017-08-13 08:54:50 -07:00
Jarek Kowalski
8dda3bf2c5
added option to read portions of blob storage blocks
2017-08-12 23:11:01 -07:00
Jarek Kowalski
94f5420557
re-added support for bundles (read-only) to aid data migration
2017-08-12 14:07:22 -07:00
Jarek Kowalski
ce4d59d8bb
added support for cancelling upload with Ctrl-C which marks the snapshot as incomplete
...
added --include-incomplete to 'backups' command
improved upload API, added support for better cancellation and upload limits
changed hashcache Reader and Writer to use interfaces
2017-08-10 20:27:31 -07:00
Jarek Kowalski
2c86e54c81
removed support for bundles now that packs are looking much better and more flexible
2017-08-10 18:24:22 -07:00
Jarek Kowalski
6821b8def4
Initial support for creating pack objects, which are identified by
...
P<pack-id>.<block-id> and store smaller files in big objects.
For each pack there is a corresponding metadata object that ObjectID to (offset,length) within a pack object.
2017-08-07 14:18:00 +02:00
Jarek Kowalski
2dfcd23ea9
added hashcache database, based on BoltDB which caches results of computing ObjectID for any given os.FileInfo.
2017-08-04 13:19:28 +02:00
Jarek Kowalski
21a1cbd0bb
renamed auth.Options to auth.SecurityOptions
2017-08-01 17:47:28 +02:00
Jarek Kowalski
cfbef13aff
Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
...
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4
removed 'client' package and merged with 'repo', names still not final
2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc
replaced use of Vault with Repository
2017-08-01 08:29:41 +02:00
Jarek Kowalski
c3f0d5ddc4
moved all Vault types to 'repo' package
2017-08-01 08:29:41 +02:00
Jarek Kowalski
6e28a8688c
changed blob.Storage.ListBlock() to return cancellation function
2017-07-01 12:03:42 -07:00
Jarek Kowalski
0195d2740a
moved directory JSON format handling to internal/dir
2017-01-30 20:04:13 -08:00
Jarek Kowalski
4b89c95721
moved hashcache to internal
2017-01-29 07:50:49 -08:00
Jarek Kowalski
a5cf717397
Refactored kopia.Connection to expose snapshot.Manager, which exposes methods to manipulate snapshots.
2017-01-28 21:24:01 -08:00
Jarek Kowalski
b21a53bccd
added backup size output
2016-10-01 10:19:38 -07:00
Jarek Kowalski
88e3366a41
Added support for upload and download throttling for GCS.
2016-09-19 20:38:25 -07:00
Jarek Kowalski
3323d17536
renamed storage.Storage to blob.Storage
2016-09-08 21:11:50 -07:00
Jarek Kowalski
baa8451403
added kopia.Connection as top-level API.
...
simplified vault API
2016-09-05 22:30:27 -07:00
Jarek Kowalski
7d066ae2ea
godoc
2016-08-28 10:18:16 -07:00
