Jarek Kowalski
9893a552e2
metrics: disable google analytics as it's not providing any useful data, need to rethink the story here
2019-08-17 16:13:30 -07:00
Jarek Kowalski
671a63aa24
travis: generate repository snapshot for long-term storage
...
This is triggered for tagged releases and will be used later for
long-term repository compatibility testing.
2019-08-12 19:32:25 -07:00
Jarek Kowalski
b365d3414c
removed content.Manager.ListContents*() API
2019-07-27 19:10:58 -07:00
Jarek Kowalski
8ada9587f3
travis: added encrypted 'known_hosts' file for sftp testing
2019-07-16 20:29:39 -10:00
Jarek Kowalski
59f0231f4c
minor testability tweaks
2019-07-16 15:48:00 -10:00
Jarek Kowalski
769b3c5f12
travis: reencrypted credentials with fixed key/iv
2019-07-14 19:30:22 -07:00
Jarek Kowalski
975824768e
sftp: added encrypted e2e test credentials
2019-07-14 10:25:10 -07:00
Jarek Kowalski
e414e7a4d1
Added support for reconnect tokens
...
Repository.Token() generates a base64-encoded token that can
be stored in password manager that fully describes repository connection
information (blob.ConnectionInfo) and optionally a password.
Use `kopia repo status -t` to print the token.
Use `kopia repo status -t -s` to print the token that also includes
repository password.
Use `kopia repo connect from-config --token T` to reconnect using the
token.
2019-07-10 21:03:40 -07:00
Jarek Kowalski
b16e19dc64
fixing gcs credentials, try #3 - regenerated new SA JSON key
2019-07-04 10:56:50 -07:00
Jarek Kowalski
cb7127ac4a
fixed credentials path, try #2
2019-07-04 10:20:25 -07:00
Jarek Kowalski
27f5b0540d
fixed installation of GCS credentials for testing, which was silently being ignored
2019-07-04 10:13:21 -07:00
Jarek Kowalski
72520029b0
golangci-lint: added more linters
...
Also fixed pre-existing lint errors.
2019-06-02 22:56:57 -07:00
Jarek Kowalski
54edb97b3a
refactoring: renamed repo/block to repo/content
...
Also introduced strongly typed content.ID and manifest.ID (instead of string)
This aligns identifiers across all layers of repository:
blob.ID
content.ID
object.ID
manifest.ID
2019-06-01 22:24:19 -07:00
Jarek Kowalski
916da07e0f
deprecate block format v0
2019-06-01 16:40:51 -07:00
Jarek Kowalski
9e5d0beccd
refactoring: renamed storage.Storage to blob.Storage
...
This updates the terminology everywhere - blocks become blobs and
`storage.Storage` becomes `blob.Storage`.
Also introduced blob.ID which is a specialized string type, that's
different from CABS block ID.
Also renamed CLI subcommands from `kopia storage` to `kopia blob`.
While at it introduced `block.ErrBlockNotFound` and
`object.ErrObjectNotFound` that do not leak from lower layers.
2019-06-01 14:10:35 -07:00
Jarek Kowalski
1a7a02ddbe
cleanup imports by grouping all local imports together
2019-06-01 10:57:55 -07:00
Jarek Kowalski
63303904e1
switched remaining fmt.Errorf to errors.Wrap()
2019-06-01 10:57:05 -07:00
Jarek Kowalski
0c41d41276
Fixed up paths after merge
2019-05-27 15:48:39 -07:00
Jarek Kowalski
b6abef96e6
Merge github.com:kopia/repo into import-repo
2019-05-27 15:41:55 -07:00
Jarek Kowalski
bdafe117d9
Makefile: switched linter to golangci-lint and updated goveralls setup
...
fixed lint errors & removed .gometalinter config
2019-04-01 19:22:01 -07:00
Jarek Kowalski
fdfd336d6e
lint: fixed various linter issues
2019-04-01 18:59:48 -07:00
Jarek Kowalski
913f3392e3
cli: renamed 'repo migration' to 'snapshot migrate' and improved caching
2019-01-19 10:52:03 -08:00
Jarek Kowalski
b37c29513b
cli: added 'repo repair' subcommand which operates on the same arguments as 'repo connect', but it fixes contents of the repository by restoring missing kopia.repository file
...
tests: reorganized end-to-end tests
2019-01-19 05:48:40 -05:00
Jarek Kowalski
c35037601d
repo: embed format block bytes in all pack indexes
...
This allows format block to be recovered if any block
indexes survives. The block is not encrypted, but has HMAC is
followed by 2-byte size.
2019-01-12 14:24:09 -08:00
Jarek Kowalski
2e297dc60d
repo: upgraded to latest version, removed format block settings from repository create, instead --encryption=NONE disables encryption of both blocks and format
2019-01-12 09:24:12 -08:00
Jarek Kowalski
000ca67f25
test: fixed end_to_end_test
2019-01-02 19:59:21 -08:00
Jarek Kowalski
840d5ab749
removed support for legacy block format, to migrate sync to previous commit and run 'kopia repo upgrade'
2018-12-31 19:01:08 -08:00
Jarek Kowalski
6e8a48ed4f
Revamped hash and encryption algorithm handling.
...
This enables many more combinations of hash and encryption algorithm
to be selected, some with dramatically better performance.
2018-12-29 15:08:29 -08:00
Jarek Kowalski
afad2ad237
repo: moved repository_test to top level
2018-12-23 20:53:44 -08:00
Jarek Kowalski
3487cc191e
removed asynchronous mode for object.Manager
2018-12-22 19:58:00 -08:00
Jarek Kowalski
e458ee24d8
imported github.com/kopia/kopia/repo and renamed package path to github.com/kopia/repo/
2018-10-26 17:33:58 -07:00
Jarek Kowalski
bcbab79dd0
coverage: updated 'go test' command to include all packages
...
previously the coverage for each package only included lines covered by tests in the same package,
excluding end-to-end tests
now, we get very different coverage numbers...
2018-10-20 09:20:08 -07:00
Jarek Kowalski
3d61b4b39b
cli: fixed some 'diff' issues and added integration test
2018-10-06 11:32:54 -07:00
Jarek Kowalski
fbffab8bc9
cli: create global policy when repository is created
2018-09-29 09:51:15 +02:00
Jarek Kowalski
1bbc169c0d
added missing package godoc, fixed test paths
2018-08-31 19:12:58 -07:00
Jarek Kowalski
91066f2469
reorganized low-level repository packages by moving them all under kopia/kopia/repo/
2018-08-30 22:01:05 -07:00
Jarek Kowalski
0f20965143
added 'blockindex recover' tests
2018-08-25 16:30:11 -07:00
Jarek Kowalski
be32f6b717
enabled more comprehensive repository stress test after resolving remaining races:
...
skipped stress tests during short testing
changed default make target to run short tests
2018-07-20 18:48:00 -08:00
Jarek Kowalski
09c80ed0e9
disabled repository stress
2018-07-09 22:17:11 -07:00
Jarek Kowalski
c57531b751
switched Go logging library to github.com/op/go-logging which lets us override log levels per module
2018-07-09 21:28:12 -07:00
Jarek Kowalski
1c99dae4b1
added new type of stress test that utilizes multiple repository features
2018-07-09 21:28:12 -07:00
Jarek Kowalski
53db414ff7
Added simple analytics mechanism based on Google Analytics for tracking features usage and latency.
...
Controlled on command line via --analytics-consent (defaults to asking user)
2018-05-30 21:22:07 -07:00
Jarek Kowalski
d6477377f9
changed how index compaction works:
...
instead of having time-based naming, block manager will perform occasional compaction at startup time and delete unwanted blocks
The protocol is safe from concurrency standpoint and provides eventual consistency.
We only delete blocks after creating compacted blocks.
We retry loading index until (any) consistent index is fetched (possibly from cache) and all underlying blocks are also available, not necessarily the latest ones.
TODO - we need to periodically snapshot block index contents, so that if we have a bug somewhere in compaction code, we have a way of restoring working indexes.
2018-05-15 20:10:50 -07:00
Jarek Kowalski
6f7b3b8a80
fixed stress test failures due to expiration of a context
2018-05-09 22:22:37 -07:00
Jarek Kowalski
3a02ef24e0
retire block.ContentID and replace with string
2018-05-05 16:16:48 -07:00
Jarek Kowalski
be3c4e725e
added stress test
2018-05-05 16:14:31 -07:00
Jarek Kowalski
52bcb6223f
when loading active index, load 10 minutes extra to correct for clock skew and in-flight uploads
2018-04-21 10:36:53 -07:00
Jarek Kowalski
ab4fa93c92
remove cache directory on disconnect
2018-03-19 12:26:28 -07:00
Jarek Kowalski
6707d01a22
switched local caching to use storage.Storage
2018-03-04 16:58:20 -08:00
Jarek Kowalski
e710dcdff3
additional integration tests
2018-02-18 11:59:32 -08:00
