Jarek Kowalski
3e36ce748f
removed conditional PutBlock, which was causing more trouble that it's worth
...
with unconditional rewrites any partial block writes are self-correcting on subsequent updates,
without it they can't complete at all.
2017-08-13 08:54:50 -07:00
Jarek Kowalski
8dda3bf2c5
added option to read portions of blob storage blocks
2017-08-12 23:11:01 -07:00
Jarek Kowalski
94f5420557
re-added support for bundles (read-only) to aid data migration
2017-08-12 14:07:22 -07:00
Jarek Kowalski
ce4d59d8bb
added support for cancelling upload with Ctrl-C which marks the snapshot as incomplete
...
added --include-incomplete to 'backups' command
improved upload API, added support for better cancellation and upload limits
changed hashcache Reader and Writer to use interfaces
2017-08-10 20:27:31 -07:00
Jarek Kowalski
2c86e54c81
removed support for bundles now that packs are looking much better and more flexible
2017-08-10 18:24:22 -07:00
Jarek Kowalski
6821b8def4
Initial support for creating pack objects, which are identified by
...
P<pack-id>.<block-id> and store smaller files in big objects.
For each pack there is a corresponding metadata object that ObjectID to (offset,length) within a pack object.
2017-08-07 14:18:00 +02:00
Jarek Kowalski
2dfcd23ea9
added hashcache database, based on BoltDB which caches results of computing ObjectID for any given os.FileInfo.
2017-08-04 13:19:28 +02:00
Jarek Kowalski
21a1cbd0bb
renamed auth.Options to auth.SecurityOptions
2017-08-01 17:47:28 +02:00
Jarek Kowalski
cfbef13aff
Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
...
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4
removed 'client' package and merged with 'repo', names still not final
2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc
replaced use of Vault with Repository
2017-08-01 08:29:41 +02:00
Jarek Kowalski
c3f0d5ddc4
moved all Vault types to 'repo' package
2017-08-01 08:29:41 +02:00
Jarek Kowalski
6e28a8688c
changed blob.Storage.ListBlock() to return cancellation function
2017-07-01 12:03:42 -07:00
Jarek Kowalski
0195d2740a
moved directory JSON format handling to internal/dir
2017-01-30 20:04:13 -08:00
Jarek Kowalski
4b89c95721
moved hashcache to internal
2017-01-29 07:50:49 -08:00
Jarek Kowalski
a5cf717397
Refactored kopia.Connection to expose snapshot.Manager, which exposes methods to manipulate snapshots.
2017-01-28 21:24:01 -08:00
Jarek Kowalski
b21a53bccd
added backup size output
2016-10-01 10:19:38 -07:00
Jarek Kowalski
88e3366a41
Added support for upload and download throttling for GCS.
2016-09-19 20:38:25 -07:00
Jarek Kowalski
3323d17536
renamed storage.Storage to blob.Storage
2016-09-08 21:11:50 -07:00
Jarek Kowalski
baa8451403
added kopia.Connection as top-level API.
...
simplified vault API
2016-09-05 22:30:27 -07:00
Jarek Kowalski
7d066ae2ea
godoc
2016-08-28 10:18:16 -07:00
Jarek Kowalski
6f5b5dc9a9
Enabled bundling by default.
2016-08-27 14:29:35 -07:00
Jarek Kowalski
940e63832d
Moved ObjectID and BundledChildren out of EntryMetadata and into repofs.dirEntry.
...
Fixed metadata hashing of bundles.
2016-08-27 10:45:45 -07:00
Jarek Kowalski
cded7ba49b
renamed fs.EntryMetadataReadCloser as fs.Reader
2016-08-26 21:31:51 -07:00
Jarek Kowalski
38fc47e2f1
Continuing the API cleanup, Created internal/dirstream, internal/mockfs, repo/repofs and upload packages.
2016-08-26 21:26:35 -07:00
Jarek Kowalski
bf40d91585
replaced storage.BlockExists() with storage.BlockSize() which returns additional info
2016-08-25 21:39:17 -07:00
Jarek Kowalski
381d3780dc
simplified Storage.PutBlock() API - passing []byte instead of convoluted reader, this greatly simplifies crypto code in the Repository
...
removed Storage.Flush()
2016-08-24 22:18:22 -07:00
Jarek Kowalski
dcd7e0f76f
godoc cleanup, hiding internal APIs
2016-08-21 08:28:40 -07:00
Jarek Kowalski
91d4346f67
replaced generated protobuf code with hand-written, switched back to JSON serialization, but preserving protobuf code structure.
...
The generated code leaked unclean concepts to public API.
2016-08-21 00:21:40 -07:00
Jarek Kowalski
e528b32748
Major refactoring of data formats to use protobuf instead of JSON. This is a breaking change to data formats, but hoping to stabilize soon.
...
This change removes lots of pointless string and custom format parsing code and instead relies on protobuf to do the thing. JSON is still an option thanks to proto3-generated output.
- Refactored ObjectID to use protobuf
- Indirect blocks to use protobuf (in the form of seek table)
- Added generic proto stream reader and writer
- ObjectIDFormat became proto enum
Also:
- fixed major issue where indirect object ID entries were not encrypted, since they were storing block IDs and not object IDs.
- dropped support for majority of formats, only supporting HMAC-{SHA256,SHA512,SHA512_384} hashes with AES256.
This produces object IDs of 32 or 64 characters long.
- changed how pretty-printing works for 'kopia show'
2016-08-20 16:59:10 -07:00
