mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-05-13 17:25:13 -04:00
Code Issues Packages Projects Releases Wiki Activity
2,239 Commits 7 Branches 89 Tags
f5f2be5116a03cc0509ffbac09b8a4ac1717731f
Commit Graph

2239 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot]
f5f2be5116 build(deps): bump plist from 3.0.4 to 3.0.5 in /app (#1851) 2022-03-26 19:42:52 +00:00
Jarek Kowalski
92683c5a5e feat(snapshots): support for controlling upload parallelism via policies (#1850) 2022-03-26 19:20:49 +00:00
Jarek Kowalski
d7e10dba59 feat(cli): improved kopia benchmark commands (#1849) 
* added --parallel flag
* added `kopia benchmark encryption`
* added `kopia benchmark hashing`
 2022-03-26 14:28:08 +00:00
Jarek Kowalski
b2b5e6fb97 fix(repository): fixed 'unable to get content data and info' (#1844) 
We were incorrectly assigning the same content cache key regardless of
the storage format of the content (compression format, etc.)

This means that if a content has both compressed and non-compressed
entry in the index, it would sometimes get cached incorrectly
and the subsequent reads would fail.

Clearing the cache would fix the issue - this change appends
format-specific suffix to cache keys, so that clearing the cache is not
needed.

Huge thanks to Mark Derricutt who helped get to the bottom of this
on Slack.

Fix #1843
 2022-03-22 20:39:36 -07:00
Ali Dowair
aafe56cd6f feat(snapshots): support restoring sparse files (#1823) 
* feat(snapshots): support restoring sparse files

This commit implements basic support for restoring sparse files from
a snapshot. When specifying "--mode=sparse" in a snapshot restore
command, Kopia will make a best effort to make sure the underlying
filesystem allocates the minimum amount of blocks needed to persist
restored files. In other words, enabling this feature will "force"
all restored files to be sparse-blocks of zero bytes in the source
file should not be allocated.

* Address review comments

- Separate sparse option into its own bool flag
- Implement sparsefile packagewith copySparse method
- Truncate once before writing sparse file
- Check error from Truncate
- Add unit test for copySparse
- Invoke GetBlockSize once per file copy
- Remove support for Windows and explain why
- Add unit test for stat package

Co-authored-by: Dave Smith-Uchida <dave@kasten.io>
 2022-03-22 19:09:50 -07:00
Jarek Kowalski
dc0d41b7e0 build(deps): upgrade minimist to 1.2.6 (#1845) 2022-03-21 21:50:14 -07:00
Jarek Kowalski
716ed71b59 fix(server): fixed server startup race condition (#1842) 
This fixes a regression introduced in #1837
 2022-03-20 22:21:03 -07:00
Jarek Kowalski
3b45daf27a fix(ci): fixed TestThrottler flake (#1841) 2022-03-20 20:38:32 -07:00
Jarek Kowalski
3ef472248d fix(ci): fixed flaky TestServerControl (#1840) 2022-03-20 20:19:21 -07:00
Jarek Kowalski
daa62de3e4 chore(ci): added checklocks static analyzer (#1838) 
From https://github.com/google/gvisor/tree/master/tools/checklocks

This will perform static verification that we're using
`sync.Mutex`, `sync.RWMutex` and `atomic` correctly to guard access
to certain fields.

This was mostly just a matter of adding annotations to indicate which
fields are guarded by which mutex.

In a handful of places the code had to be refactored to allow static
analyzer to do its job better or to not be confused by some
constructs.

In one place this actually uncovered a bug where a function was not
releasing a lock properly in an error case.

The check is part of `make lint` but can also be invoked by
`make check-locks`.
 2022-03-19 22:42:59 -07:00
Jarek Kowalski
9cba4a97be refactor(repository): major server code refactoring (#1837) 
This removes big shared lock held for for the duration of each request
and replaces it with trivially short lock to capture the current
state of the server/repository before passing it to handlers.

Handlers are now limited to only accessing a small subset of Server
functionality to be able to better reason about them.
 2022-03-19 22:01:38 -07:00
Jarek Kowalski
9054fd0dc2 chore(ci): upgraded linter to 1.45 (#1836) 
* chore(deps): upgraded linter to 1.45

* fixed linter warning
 2022-03-18 22:24:42 -07:00
Jarek Kowalski
6ba2f586ad feat(repository): ensure we don't run parallel fetches for the same blob IDs (#1835) 
* feat(repository): ensure we don't run parallel fetches for the same blob IDs

Fixed a bunch of test flakes.

* fixed race condition, limit the size of mutex cache by using LRU
 2022-03-18 21:27:06 -07:00
Jarek Kowalski
991c08e4b4 chore(repository): switched from opencensus to directly exporting prometheus metrics (#1831) 2022-03-17 23:39:36 -07:00
Julio Lopez
c6e39878b3 refactor(repository): minor refactor for common code path in helper (#1833) 2022-03-17 22:25:23 -07:00
Julio Lopez
3f9e27c97b feat(cli): add --json output to 'repo status' command (#1834) 
* cleanup(repository) add kopia:sensitive tag to content.FormattingOptions field

* feat(cli): add --json output to  command
 2022-03-17 22:22:24 -07:00
Ali Dowair
c7ce87f95b feat(providers): Implement API to get Storage free space (#1753) 
* Introduce Volume sub-interface
  The Volume interface defines APIs to access a storage provider's
  volume (disk) capacity, usage, etc.. It is inherited by the Storage
  interface, and is at the same hierarchical level as the Reader
  interface.

* Add validations for new Volume method:
  Check that GetCapacity() either returns `ErrNotAVolume`, or that it
  returns a Capacity struct with values that make sense.

* Implement default (passthrough) GetCapacity:
  Cloud providers do not have finite volumes, and WebDAV volumes have no
  notion of volume size and usage. These implementations should just
  return an error (ErrNotAVolume) when their GetCapacity() is called.

* Implement GetCapacity for sftp storage: Uses the sftp.Client interface
* Implement GetCapacity for logging, readonly store
* Implement GetCapacity() for blobtesting implementations

* Implement GetCapacity() for Google Drive:
  Also modifies GetDriveClient to return the entire service instead of just the Files client.

* Implemented GetCapacity() for filesystem storage:
  Implemented the function in a seperate file for each OS/architecture (Unix, OpenBSD, Windows).
 2022-03-16 12:35:33 -07:00
Jarek Kowalski
3ef81134a1 fix(cli): fixing ignoring lines starting with '#' in 'policy edit' (#1830) 
Fixes #1821
 2022-03-16 08:00:50 -07:00
Jarek Kowalski
2b8f7453db fix(cli): fixed and unified help text for policy commands (#1829) 
Fixes #1822
 2022-03-16 00:46:57 -07:00
Jarek Kowalski
c4ab086d5f chore(repository): streamlined internal content cache API (#1828) 
moved from ./repo/content to ./internal/cache
 2022-03-15 23:57:04 -07:00
Jarek Kowalski
1aa3a35518 refactor(repository): moved index blob cache to separate directory (#1827) 
Also cleaned up internal content cache APIs.
 2022-03-13 21:40:15 -07:00
Jarek Kowalski
69dc7ba969 feat(repository): added 'hint' to Prefetch methods. (#1825) 2022-03-12 23:16:39 -08:00
Jarek Kowalski
db7dcac33c feat(repo): exposed PrefetchContents on the repo.Repository() (#1824) 2022-03-12 14:36:54 -08:00
Jarek Kowalski
e591f4b79b chore(ci): skip non-deterministic tests when computing code coverage (#1817) 2022-03-09 07:42:01 -08:00
Jarek Kowalski
68c0d6e278 fix(snapshots): fixed snapshotting of mounted VSS snapshot roots (#1816) 2022-03-08 21:32:11 -08:00
Jarek Kowalski
c6944c70b1 fix(repository): fix deduplication when snapshotting identical files in parallel (#1815) 
This issue was reported on Slack by Ming. Thanks!
This is particularly easy to reproduce when compression is used.
 2022-03-08 20:11:06 -08:00
Jarek Kowalski
766cb57160 feat(ci): automatically bump homebrew version on tag (#1807) v0.10.6 2022-03-06 21:44:13 -08:00
Jarek Kowalski
53bc4d13f4 feat(ui): display snapshot description in snapshot list (#1806) 
Fixes #1769
 2022-03-06 21:09:20 -08:00
Jarek Kowalski
436933caa1 feat(kopiaui): support both left and right click to open tray menu (#1805) 
* feat(kopiaui): support both left and right click to open tray menu

Fixes #1798

* feat(kopiaui): show all repository windows on double click
 2022-03-06 21:09:01 -08:00
Jarek Kowalski
353e8b1ff4 feat(kopiaui): provide feedback for 'Check For Updates Now' (#1804) 
Fixes #1797
 2022-03-06 20:25:27 -08:00
Jarek Kowalski
133513c2cb feat(repository): removed noisy log output (#1803) 2022-03-06 18:51:23 -08:00
Jarek Kowalski
369d304084 refactor(repository): better context cancelation handling (#1802) 
Instead of ignoring context cancelation in Open(), ensure we don't
spawn goroutines that might be canceled.
 2022-03-06 16:56:30 -08:00
dependabot[bot]
a70da61d0e build(deps-dev): bump electron from 17.0.0 to 17.1.0 in /app (#1784) 
Bumps [electron](https://github.com/electron/electron) from 17.0.0 to 17.1.0.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v17.0.0...v17.1.0)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-06 15:32:29 -08:00
dependabot[bot]
526b1d7377 build(deps): bump go.uber.org/zap from 1.20.0 to 1.21.0 (#1795) 
Bumps [go.uber.org/zap](https://github.com/uber-go/zap) from 1.20.0 to 1.21.0.
- [Release notes](https://github.com/uber-go/zap/releases)
- [Changelog](https://github.com/uber-go/zap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/uber-go/zap/compare/v1.20.0...v1.21.0)

---
updated-dependencies:
- dependency-name: go.uber.org/zap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-06 14:31:34 -08:00
Jarek Kowalski
926e14aacb feat(repository): added PrefetchObjects() API (#1779) 
* feat(repository): added precaching of data blobs

* feat(repository): added utilities for converting ID slices to strings

* feat(repository): added object.PrefetchBackingContents

* feat(repository): implemented Repository.PrefetchObjects

* feat(cli): added 'cache prefetch' subcommand

* feat(repository): prefetch in parallel

* added tests
 2022-03-06 14:30:58 -08:00
Jarek Kowalski
4962c5f501 chore(repository): added context to beforeop callbacks (#1800) 
This helps eliminate a class of bugs where 'ctx' used in a callbacks
is captured from a closure variable/parameter, which is almost never
correct, because it can be canceled while the beforeop wrapper is
still alive.

Follow-up for #1796.
 2022-03-06 12:02:36 -08:00
Jarek Kowalski
6baf10ad31 fix(repository): fixed 'context canceled' regression (#1799) 
This was broken in #1758 by holding onto a context beyond its intended
lifetime.

Fixes #1796
 2022-03-03 21:02:40 -08:00
dependabot[bot]
192191e056 build(deps): bump github.com/aws/aws-sdk-go from 1.42.44 to 1.43.9 (#1785) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.42.44 to 1.43.9.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.42.44...v1.43.9)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-02 20:12:32 -08:00
dependabot[bot]
5f0dc23d25 build(deps): bump google.golang.org/api from 0.69.0 to 0.70.0 (#1791) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.69.0 to 0.70.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.69.0...v0.70.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-02 20:12:20 -08:00
dependabot[bot]
5932a660e6 build(deps): bump github.com/minio/minio-go/v7 from 7.0.21 to 7.0.23 (#1794) 
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.21 to 7.0.23.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.21...v7.0.23)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-02 19:31:17 -08:00
dependabot[bot]
8f778ce951 build(deps): bump github.com/chromedp/chromedp from 0.7.6 to 0.7.8 (#1793) 
Bumps [github.com/chromedp/chromedp](https://github.com/chromedp/chromedp) from 0.7.6 to 0.7.8.
- [Release notes](https://github.com/chromedp/chromedp/releases)
- [Commits](https://github.com/chromedp/chromedp/compare/v0.7.6...v0.7.8)

---
updated-dependencies:
- dependency-name: github.com/chromedp/chromedp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 22:25:16 -08:00
dependabot[bot]
1617d51315 build(deps): bump github.com/klauspost/compress from 1.14.2 to 1.14.4 (#1792) 
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.14.2 to 1.14.4.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.14.2...v1.14.4)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 22:03:23 -08:00
dependabot[bot]
1d50a343f7 build(deps): bump nanoid from 3.1.30 to 3.3.1 in /app (#1790) 
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.30 to 3.3.1.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.30...3.3.1)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:26:00 -08:00
dependabot[bot]
7d5abf87f4 build(deps): bump electron-updater from 4.6.1 to 4.6.5 in /app (#1781) 
Bumps [electron-updater](https://github.com/electron-userland/electron-builder/tree/HEAD/packages/electron-updater) from 4.6.1 to 4.6.5.
- [Release notes](https://github.com/electron-userland/electron-builder/releases)
- [Changelog](https://github.com/electron-userland/electron-builder/blob/master/packages/electron-updater/CHANGELOG.md)
- [Commits](https://github.com/electron-userland/electron-builder/commits/electron-updater@4.6.5/packages/electron-updater)

---
updated-dependencies:
- dependency-name: electron-updater
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:23:50 -08:00
dependabot[bot]
dc52d624a4 build(deps): bump go.uber.org/multierr from 1.7.0 to 1.8.0 (#1783) 
Bumps [go.uber.org/multierr](https://github.com/uber-go/multierr) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/uber-go/multierr/releases)
- [Changelog](https://github.com/uber-go/multierr/blob/master/CHANGELOG.md)
- [Commits](https://github.com/uber-go/multierr/compare/v1.7.0...v1.8.0)

---
updated-dependencies:
- dependency-name: go.uber.org/multierr
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:23:17 -08:00
dependabot[bot]
c7ccf4c79d build(deps): bump actions/checkout from 2 to 3 (#1782) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:22:55 -08:00
dependabot[bot]
1005ba0ec4 build(deps): bump github.com/golang-jwt/jwt/v4 from 4.2.0 to 4.3.0 (#1787) 
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:21:29 -08:00
dependabot[bot]
ff8d64c1a9 build(deps): bump electron-log from 4.4.5 to 4.4.6 in /app (#1788) 
Bumps [electron-log](https://github.com/megahertz/electron-log) from 4.4.5 to 4.4.6.
- [Release notes](https://github.com/megahertz/electron-log/releases)
- [Changelog](https://github.com/megahertz/electron-log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/megahertz/electron-log/compare/v4.4.5...v4.4.6)

---
updated-dependencies:
- dependency-name: electron-log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:21:11 -08:00
dependabot[bot]
de0f7c4dfb build(deps): bump cloud.google.com/go/storage from 1.19.0 to 1.21.0 (#1789) 
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.19.0 to 1.21.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.19.0...spanner/v1.21.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-03-01 21:20:58 -08:00
Ali Dowair
7bc32c485a Fix failed robustness job script if TEST_RC unset (#1780) 
If the TEST_RC environment variable is unset, the robustness_job.sh
script exits prematurely due to an unbound variable error, instead of
starting the robustness job without that argument. This commit
evaluates said variable conditionally so that the script doesn't fail.
 2022-03-01 13:32:36 -08:00