Jarek Kowalski
fe70b1e8b0
re-added conditional PutObject
2017-08-12 16:42:13 -07:00
Jarek Kowalski
9d1fb5032d
improved GCS exponential backoff
2017-08-12 16:28:01 -07:00
Jarek Kowalski
60654e125b
added 'migrate' command that can move all snapshot data between repositories. It's meant to be used in a cloud VM to (relatively) quickly re-create the contents of all snapshots using new repository format. Copying is done using snapshot.Uploader on top of a virtual filesystem from the source repository.
2017-08-12 15:23:51 -07:00
Jarek Kowalski
94f5420557
re-added support for bundles (read-only) to aid data migration
2017-08-12 14:07:22 -07:00
Jarek Kowalski
634302701f
changed default pack file size to 20MB and pack item to 4MB
2017-08-10 22:31:13 -07:00
Jarek Kowalski
ce4d59d8bb
added support for cancelling upload with Ctrl-C which marks the snapshot as incomplete
...
added --include-incomplete to 'backups' command
improved upload API, added support for better cancellation and upload limits
changed hashcache Reader and Writer to use interfaces
2017-08-10 20:27:31 -07:00
Jarek Kowalski
a153c80064
tweaks to pack index format
2017-08-10 18:30:09 -07:00
Jarek Kowalski
2c86e54c81
removed support for bundles now that packs are looking much better and more flexible
2017-08-10 18:24:22 -07:00
Jarek Kowalski
a0b8a75133
changed kopia metadata show to pretty-print JSON by default since all data in the vault is JSON
2017-08-10 18:12:53 -07:00
Jarek Kowalski
e76243e550
introduced multiple packs per upload - directory data is stored together, which allows is to be cached together speeding up browsing of backups
2017-08-10 18:08:34 -07:00
Jarek Kowalski
036ce15a88
move throttling to GCS storage options and create/connect flags
2017-08-10 17:29:45 -07:00
Jarek Kowalski
e3e095c347
swapped order of packed object IDs
2017-08-09 20:13:00 -07:00
Jarek Kowalski
9ba224854e
fixed throttling in GCS provider
2017-08-09 19:47:14 -07:00
Jarek Kowalski
19339345e9
cleaned up connect/create CLI options
2017-08-09 16:59:53 -07:00
Jarek Kowalski
f9a0a5dfaf
fix for create incorrectly handling store errors
2017-08-09 08:14:10 -07:00
Jarek Kowalski
4912dc0f79
simplified pack manager
2017-08-08 08:03:43 +02:00
Jarek Kowalski
cbb492ea09
removed buffer manager
2017-08-08 07:56:39 +02:00
Jarek Kowalski
65d188ec59
removed unnecessary optimization to avoid hashing directories, this is now unnecessary since they end up being packed most of the time so they don't incur the cost of a round-trip
...
disabled bundling of entries, since support for bundles will be removed soon
2017-08-08 07:56:28 +02:00
Jarek Kowalski
22fe31044c
enable packing when uploading
2017-08-07 15:12:01 +02:00
Jarek Kowalski
dbeabbb1ef
deprecated inline object IDs, which cause significant slowdowns to the backup process, will be much more efficient with pack files
2017-08-07 15:11:01 +02:00
Jarek Kowalski
8d714b48da
parsing of pack IDs
2017-08-07 14:31:28 +02:00
Jarek Kowalski
6821b8def4
Initial support for creating pack objects, which are identified by
...
P<pack-id>.<block-id> and store smaller files in big objects.
For each pack there is a corresponding metadata object that ObjectID to (offset,length) within a pack object.
2017-08-07 14:18:00 +02:00
Jarek Kowalski
2dfcd23ea9
added hashcache database, based on BoltDB which caches results of computing ObjectID for any given os.FileInfo.
2017-08-04 13:19:28 +02:00
Jarek Kowalski
21a1cbd0bb
renamed auth.Options to auth.SecurityOptions
2017-08-01 17:47:28 +02:00
Jarek Kowalski
df83cbea27
added status subcommand
2017-08-01 17:47:27 +02:00
Jarek Kowalski
cfbef13aff
Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
...
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4
removed 'client' package and merged with 'repo', names still not final
2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc
replaced use of Vault with Repository
2017-08-01 08:29:41 +02:00
Jarek Kowalski
50c2cd86a3
refactored credentials to separate package
2017-08-01 08:29:41 +02:00
Jarek Kowalski
d8b2ba4c24
moved most methods from Repository to casManager
2017-08-01 08:29:41 +02:00
Jarek Kowalski
5721ba8140
renamed repository.go to cas_manager.go in preparation for name flip
2017-08-01 08:29:41 +02:00
Jarek Kowalski
ef71993cde
stale comment
2017-08-01 08:29:41 +02:00
Jarek Kowalski
cda3086dea
cleanup: moved writeback to separate file
2017-08-01 08:29:41 +02:00
Jarek Kowalski
c3f0d5ddc4
moved all Vault types to 'repo' package
2017-08-01 08:29:41 +02:00
Jarek Kowalski
cab2723443
removed vault object ID storage, which was really only there to make convergent encryption UI not suck as much
2017-08-01 08:29:41 +02:00
Jarek Kowalski
989bb27f06
renamed vault.Format to vault.VaultFormat in preparation for merging with repo
2017-08-01 08:29:41 +02:00
Jarek Kowalski
dd7f78468d
removed support for non-colocated repository
2017-08-01 08:29:40 +02:00
Jarek Kowalski
af62a902a9
gitignore
2017-08-01 08:29:40 +02:00
Jarek Kowalski
570dc0a897
removed support for per-objectID encryption
2017-07-18 09:23:48 +02:00
Jarek Kowalski
4a878738a0
moved main executable package to kopia/kopia, moved cli support to kopia/kopia/cli
2017-07-17 15:20:32 +02:00
Jarek Kowalski
700a4a8c92
moved Connection and ConnectionOptions to its own package
2017-07-17 14:23:15 +02:00
Jarek Kowalski
5b9f87068f
switched GCS API to cloud.google.com/go which performs automatic retries and has additional auth options
2017-07-17 11:14:33 +02:00
Jarek Kowalski
90a6f9c032
added CLI to specify splitter and parameters
2017-07-17 11:13:32 +02:00
Jarek Kowalski
70c7af092c
added configurable splitters to repo.Format
2017-07-14 11:54:30 +02:00
Jarek Kowalski
e411b5cfb4
updated objectWriter to use objectSplitter in preparation for adding rolling hash support
2017-07-14 11:16:01 +02:00
Jarek Kowalski
be7736d51d
updated rolling hash splitter to use approximate block size instead of number of bits
2017-07-14 11:04:39 +02:00
Jarek Kowalski
e574844cd0
added objectSplitter abstraction
2017-07-13 10:17:47 +02:00
Jarek Kowalski
3624a21bec
added -r and -o options to 'kopia ls'
2017-07-03 14:35:56 -07:00
Jarek Kowalski
2d1cd9b2c3
removed objectWriter.WriteGather() and simplified objectWriter.Write()
2017-07-03 10:00:04 -07:00
Jarek Kowalski
efeae91574
fixed GetStorageBlocks to handle multi-level indirections, added test
2017-07-01 22:59:43 -07:00
