Commit Graph

168 Commits

Author SHA1 Message Date
Niraj Tolia
f32f0ece2f Switch file log level to info
Reduces multi-GB log files. Main culprits were:

- [upload.go:385] could not find cache entry for...
- [content_manager.go:790] WriteContent("...") - new
- [content_manager.go:396] adding ... length=... deleted=false
2019-08-24 21:09:39 -07:00
Jarek Kowalski
399164b3be webdav: improved error handling
- improved HTTP error handling
- added exponential back-off around gowebdav calls
- fixed blob.ListAllBlobsConsistent which was never finishing for WebDAV

Fixes #88 - 429 Too Many Requests - PROPFIND /n0a #88
Fixex #89 - Debug log clashes with interactive repo password request
2019-08-18 10:49:11 -07:00
Jarek Kowalski
3a80d945d9 lint: upgraded to 1.17.1 2019-07-19 20:00:23 -10:00
Jarek Kowalski
22170b4832 cli: changed 'cache set' subcommand to support changing individual parameters 2019-06-11 22:08:52 -07:00
Jarek Kowalski
23d979f290 deleted jsonstream 2019-06-08 22:47:05 -07:00
Jarek Kowalski
ae1510b65c Deprecate hashcache and replace it with simply walking previous snapshot directory tree.
This cleans up the code a lot and removes many ugly hacks.
The performance is pretty reasonable and with separate metadata cache it's likely to stay that way.
2019-06-08 18:01:47 -07:00
Jarek Kowalski
72520029b0 golangci-lint: added more linters
Also fixed pre-existing lint errors.
2019-06-02 22:56:57 -07:00
Jarek Kowalski
54edb97b3a refactoring: renamed repo/block to repo/content
Also introduced strongly typed content.ID and manifest.ID (instead of string)

This aligns identifiers across all layers of repository:

blob.ID
content.ID
object.ID
manifest.ID
2019-06-01 22:24:19 -07:00
Jarek Kowalski
9e5d0beccd refactoring: renamed storage.Storage to blob.Storage
This updates the terminology everywhere - blocks become blobs and
`storage.Storage` becomes `blob.Storage`.

Also introduced blob.ID which is a specialized string type, that's
different from CABS block ID.

Also renamed CLI subcommands from `kopia storage` to `kopia blob`.

While at it introduced `block.ErrBlockNotFound` and
`object.ErrObjectNotFound` that do not leak from lower layers.
2019-06-01 14:10:35 -07:00
Jarek Kowalski
1a7a02ddbe cleanup imports by grouping all local imports together 2019-06-01 10:57:55 -07:00
Jarek Kowalski
63303904e1 switched remaining fmt.Errorf to errors.Wrap() 2019-06-01 10:57:05 -07:00
Jarek Kowalski
03339c18af [breaking change] deprecated DYNAMIC splitter due to license issue
The splitter in question was depending on
github.com/silvasur/buzhash which is not licensed according to FOSSA bot

Switched to new faster implementation of buzhash, which is
unfortunately incompatible and will split the objects in different
places.

This change is be semi-breaking - old repositories can be read, but
when uploading large objects they will be re-uploaded where previously
they would be de-duped.

Also added 'benchmark splitters' subcommand and moved 'block cryptobenchmark'
subcommand to 'benchmark crypto'.
2019-05-30 22:20:45 -07:00
Jarek Kowalski
0c41d41276 Fixed up paths after merge 2019-05-27 15:48:39 -07:00
Jarek Kowalski
b6abef96e6 Merge github.com:kopia/repo into import-repo 2019-05-27 15:41:55 -07:00
Jarek Kowalski
a6a153b22e switched fmt.Errorf() to errors.Wrap() 2019-05-11 12:34:14 -07:00
Jarek Kowalski
142bfa3622 site: included logging-related flags, cleanup 2019-05-09 23:00:07 -07:00
Jarek Kowalski
bdafe117d9 Makefile: switched linter to golangci-lint and updated goveralls setup
fixed lint errors & removed .gometalinter config
2019-04-01 19:22:01 -07:00
Jarek Kowalski
fdfd336d6e lint: fixed various linter issues 2019-04-01 18:59:48 -07:00
Jarek Kowalski
2e297dc60d repo: upgraded to latest version, removed format block settings from repository create, instead --encryption=NONE disables encryption of both blocks and format 2019-01-12 09:24:12 -08:00
Jarek Kowalski
766f574973 repo: removed controls for setting format block encryption, instead when block encryption is enabled, the format block is automatically encrypted using default encryption algorithm 2019-01-12 09:14:34 -08:00
Jarek Kowalski
db9759bf3a upgraded repo 2018-12-31 19:10:21 -08:00
Jarek Kowalski
840d5ab749 removed support for legacy block format, to migrate sync to previous commit and run 'kopia repo upgrade' 2018-12-31 19:01:08 -08:00
Jarek Kowalski
0b89cbef3b cli: added 'repo upgrade' command that invokes Repository.Upgrade() 2018-12-31 17:12:30 -08:00
Jarek Kowalski
24bd5bbe1f repo: added Repository.Upgrade() API 2018-12-31 17:08:41 -08:00
Jarek Kowalski
b4c80348f8 object: removed dependency on jsonstream for parsing indirect objects 2018-12-31 14:21:47 -08:00
Jarek Kowalski
a6a9729f71 upgraded repo to v0.2.0 2018-12-29 15:14:29 -08:00
Jarek Kowalski
6e8a48ed4f Revamped hash and encryption algorithm handling.
This enables many more combinations of hash and encryption algorithm
to be selected, some with dramatically better performance.
2018-12-29 15:08:29 -08:00
Jarek Kowalski
7d03d82cd1 retry: added tests for retry loop 2018-12-23 23:20:29 -08:00
Jarek Kowalski
53b4a9abd2 jsonstream: added test cases 2018-12-23 23:04:12 -08:00
Jarek Kowalski
641b25a508 repo: added tests for Disconnect() 2018-12-23 21:12:53 -08:00
Jarek Kowalski
160d02a6eb block cache: fixed a bug where we remove wrong item from the cache, added tests 2018-12-23 11:59:17 -08:00
Jarek Kowalski
a673d36b1d lint: ran gofmt -s and fixed spelling
as recommended by goreportcard.com
2018-11-07 17:46:57 -08:00
Jarek Kowalski
88e5525436 snapshotfs: merged internal/dir into snapshotfs 2018-11-03 11:47:02 -07:00
Jarek Kowalski
45ce0b35b1 snapshot: moved dir.Entry to snapshot.DirEntry 2018-11-03 11:44:19 -07:00
Jarek Kowalski
242d012a54 removed fs.EntryMetadata and replaced fs.Entry with os.FileInfo + small additions 2018-11-03 11:28:06 -07:00
Jarek Kowalski
bb4c6674c7 snapshotfs: merged internal.Upload into snapshotfs 2018-11-03 08:39:07 -07:00
Jarek Kowalski
82653d3736 packindex: merged package into /block, removed unnecessary visibility and dead code 2018-10-31 21:14:19 -07:00
Jarek Kowalski
8dee493bf3 storagetesting: added test case for zero length 2018-10-27 12:20:00 -07:00
Jarek Kowalski
7196891d17 storagetesting: added Close() 2018-10-27 12:09:30 -07:00
Jarek Kowalski
4237bbc42a storagetesting: verify progress callbacks 2018-10-27 12:07:54 -07:00
Jarek Kowalski
4b4f9240df storage: additional storage tests, added GCS test suite 2018-10-27 11:38:51 -07:00
Jarek Kowalski
0c67743168 storagetesting: new test cases 2018-10-27 08:47:36 -07:00
Jarek Kowalski
6170e5423e fixed build break 2018-10-26 20:49:49 -07:00
Jarek Kowalski
327d8317d8 refactored repo/ into separate github.com/kopia/repo/ git repository 2018-10-26 20:40:57 -07:00
Jarek Kowalski
e458ee24d8 imported github.com/kopia/kopia/repo and renamed package path to github.com/kopia/repo/ 2018-10-26 17:33:58 -07:00
Jarek Kowalski
a824c96271 refactor: repo.NewRepositoryOptions 2018-10-23 21:37:30 -07:00
Jarek Kowalski
dd1c0943cd refactor: moved config file management to kopia/kopia/repo
also fixed layering issue and removed dependency from 'object'
on 'config'
2018-10-23 19:43:43 -07:00
Jarek Kowalski
3a4b581814 snapshot: added tests 2018-10-21 22:46:38 -07:00
Jarek Kowalski
d47f662316 refactoring: moved kopia/kopia/policy to kopia/kopia/snapshot/policy 2018-10-20 09:55:52 -07:00
Jarek Kowalski
f43eae0493 packindex: removed dead code 2018-10-19 17:26:11 -07:00