mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-03-12 11:16:25 -04:00
Code Issues Packages Projects Releases Wiki Activity
2,553 Commits 4 Branches 88 Tags
4353bfcec04577cc800770adf4038aa69fdfda93
Commit Graph

2553 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot]
4353bfcec0 build(deps): bump github.com/aws/aws-sdk-go from 1.44.84 to 1.44.92 (#2376) 2022-09-07 04:23:40 +00:00
Jarek Kowalski
9fc84bb12f fix(repository): fixed async write error (#2373) 
Fixes #2369
 2022-09-06 14:41:44 +00:00
Ricardo Pescuma Domenecci
31495392a1 feat(cli): Allow restore from snapshoted path (#2335) 
* feat(cli): Allow restore from snapshoted path

* Find files in multiple snapshots

* Added --snapshot-time to restore

* Added restore by path test

* More timespec formats

* Test for snapshot list with a file in multiple snapshots

* Handle restore without target path

* Fix for tests

* Made changes requested in PR and rebased
 2022-09-06 04:42:07 +00:00
Jarek Kowalski
afdfad6691 feat(general): move kopia binary in Docker container to /bin/kopia (#2368) 2022-09-04 21:02:15 +00:00
Jarek Kowalski
c509e13249 fix(ci): fixed TestUpload_VirtualDirectoryWithStreamingFileWithModTime/CurrentTime (#2367) 2022-09-04 18:43:54 +00:00
s-h-a-r-d
63257d9537 feat(general): add curl to Docker container image (#2363) 2022-09-03 21:30:13 +00:00
Jarek Kowalski
a3b1f69718 feat(snapshots): always serialize timestamps in UTC timezone (#2362) 
This actually fixes test runs on non-UTC timezone which were broken
by #2343.
 2022-09-03 03:44:54 +00:00
Jarek Kowalski
cbb7d68fb6 feat(repository): implemented format.Manager - take 2 (#2349) 
* implemented format blob cache abstraction

* moved upgrade lock logic to repo/format

* moved set parameters logic to repo/format

* moved change password functionality to repo/format

* mechanical changes

* mechanical changes to react to format manager interface

* get current repository format bytes instead of static

* implemented format.Manager which dynamically fetches and caches latest format blob

* repo changes to use format.Manager

* fixed failing unit test due to different timings

* reduced lock contention by using RWMutex

* serve immutable parts of format without any locks

* increase test timeout

* fixed handling of negative validDuration

The new rules are:

- validDuration < 0 - ignore initial cached file, refresh every 15min
- validDuration > 15min - refresh every 15 minutes
- validDuration > 0 && validDuration <= 15min - refresh using provided
  interval (mostly used for testing)
 2022-09-02 18:55:01 -07:00
Jarek Kowalski
7bda16ab33 feat(repository): introduced fs.UTCTimestamp (#2343) 
Fixes #2342
 2022-09-02 10:35:59 -07:00
chaitalisg
0ae4d63c61 test(general): Fix upgrade portion of robustness test (#2337) 
* Move upgrade repository test function

* Delete test for deleted function

* Move upgrade repository test function

* Resolve conflicts

* Check for error in status function

* Add unit test
 2022-08-29 10:57:37 -07:00
basldfalksjdf
9957256fef docs(site): Updating repository testing and recovery page (#2272) 
* [WIP] docs(site): Updating repository testing and recovery page

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md
 2022-08-25 10:44:58 +01:00
dependabot[bot]
6afd28968a build(deps): bump github.com/aws/aws-sdk-go from 1.44.83 to 1.44.84 (#2347) 2022-08-24 23:54:54 +00:00
dependabot[bot]
8c13027760 build(deps): bump google.golang.org/grpc from 1.48.0 to 1.49.0 (#2346) 2022-08-24 23:31:06 +00:00
dependabot[bot]
33ab6146f6 build(deps): bump github.com/aws/aws-sdk-go from 1.44.82 to 1.44.83 (#2341) 2022-08-23 23:32:44 +00:00
Niklas Hambüchen
2dd752b1df fix(cli): Fix /debug/pprof/heap returning 404 (#2339) 
See https://github.com/kopia/kopia/issues/1903#issuecomment-1127172904
 2022-08-23 08:55:25 +00:00
dependabot[bot]
dd1a6aff8d build(deps): bump github.com/aws/aws-sdk-go from 1.44.81 to 1.44.82 (#2338) 2022-08-22 23:38:00 +00:00
ashmrtn
5c88bcf1a6 feat(snapshots): Callback for when uploader finishes processing a file (#2331) 
* Make callback for upload file completion

Callback does not indicate that a file will be reachable immediately in
the resulting snapshot, but does indicate that the uploader is done
processing the file in some way (either via uploading data or finding a
previous version in the repo) and whether there was an error processing
the file.

* Tests for new FinishedFile callback

Ensure hadErr is properly populated and FinishedFile is called even if
the file was considered cached.

* Refine comment on interface function slightly

* Give callback error instead of bool about error

* Add locks around concurrent accesses in test
 2022-08-22 20:42:27 +01:00
dependabot[bot]
b97e567d06 build(deps): bump github.com/aws/aws-sdk-go from 1.44.80 to 1.44.81 (#2330) 2022-08-19 23:35:24 +00:00
Shikhar Mall
11c7bf533c feat(cli): flag to allow placing the upgrade lock without doing anything else (#2329) 
Co-authored-by: Shikhar Mall <small@kopia.io>
 2022-08-19 13:48:08 -07:00
Jarek Kowalski
5ad8f1cf38 fix(ci): try to fix HTMLUI test flake (again) (#2326) 2022-08-19 09:33:52 -07:00
dependabot[bot]
fd2969dfa5 build(deps): bump github.com/mattn/go-colorable from 0.1.12 to 0.1.13 (#2320) 
Bumps [github.com/mattn/go-colorable](https://github.com/mattn/go-colorable) from 0.1.12 to 0.1.13.
- [Release notes](https://github.com/mattn/go-colorable/releases)
- [Commits](https://github.com/mattn/go-colorable/compare/v0.1.12...v0.1.13)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-colorable
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-19 15:46:59 +00:00
Ricardo Pescuma Domenecci
04f54000a6 feat(server): Added ECC to server api (#2314) 
* feat(server): Added ECC to server api

* feat(server): Add format version to repo status api

* Change requested in PR

* Updated htmlui
 2022-08-19 08:24:04 -07:00
dependabot[bot]
1dc2fc7561 build(deps): bump github.com/aws/aws-sdk-go from 1.44.78 to 1.44.80 (#2323) 2022-08-19 15:14:12 +00:00
Ricardo Pescuma Domenecci
8b41543250 chore(cli): make ecc options visible but marked as experimental (#2322) 2022-08-19 15:04:40 +00:00
Jarek Kowalski
cf23809dcd fix(ci): fixed TestFullMaintenance on Windows (#2325) 
Unlike on other OSes it appears that on Windows time.Now() called twice
in quick succession can return the same value.

Switched logic using `time.After()` to `!time.Before()`
 2022-08-19 08:01:34 -07:00
Shikhar Mall
e317cdba9f fix: drop the ability to set advance-notices on upgrade (#2312) 
Co-authored-by: Shikhar Mall <small@kopia.io>
 2022-08-19 07:35:56 -07:00
dependabot[bot]
ff1653c4d6 build(deps): bump google.golang.org/api from 0.92.0 to 0.93.0 (#2321) 2022-08-17 06:07:05 +00:00
dependabot[bot]
5a8cde2863 build(deps): bump github.com/chromedp/chromedp from 0.8.3 to 0.8.4 (#2316) 
Bumps [github.com/chromedp/chromedp](https://github.com/chromedp/chromedp) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/chromedp/chromedp/releases)
- [Commits](https://github.com/chromedp/chromedp/compare/v0.8.3...v0.8.4)

---
updated-dependencies:
- dependency-name: github.com/chromedp/chromedp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-16 22:30:33 -07:00
dependabot[bot]
7ea63afe07 build(deps): bump github.com/aws/aws-sdk-go from 1.44.75 to 1.44.78 (#2318) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.75 to 1.44.78.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.75...v1.44.78)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-16 22:30:10 -07:00
ashmrtn
5c031b17fa feat(snapshots): Allow uploader to find cached StreamingFiles (#2317) 
* Allow setting mod time on StreamingFiles

Only set during struct creation. Default the old constructor to using
the current time as the mod time.

* Change how mod time is handled for StreamingFiles

Don't set StreamingFile mod time in the uploader, instead use the value
in the file's metadata. Also allows StreamingFiles to be recognized as
cached files (previously uploaded). StreamingFiles don't know their file
size until they've been completely uploaded so leaving that out makes
them eligible for being marked as "cached".

This commit combined with the previous commit slightly changes how
timestamps on StreamingFiles are handled. It will result in them having
slightly earlier timestamps because they are now set on struct creation
instead of when the file was uploaded.

As timestamps are fairly fine-grained and the default is to use the
current time as the mod time it seems unlikely this patch will result in
incorrectly thinking a StreamingFile is cached even though it has
changed size.

* Uploader test for StreamingFile caching
 2022-08-16 22:00:52 -07:00
Ricardo Pescuma Domenecci
47aaa2dc40 feat(cli): Added ECC related options to repository create cli command (#2308) 
* Encryptor pipeline

* Added ECC related options to repository create cli command

* Fix for lint errors

* Fixing comments from the PR

* Fixed lint errors

* Changes requested in PR

* Created e2e test
 2022-08-13 08:49:22 -07:00
dependabot[bot]
5fd7c62eca build(deps): bump github.com/aws/aws-sdk-go from 1.44.73 to 1.44.75 (#2310) 2022-08-11 23:38:40 +00:00
dependabot[bot]
3b8c36f830 build(deps): bump cloud.google.com/go/storage from 1.24.0 to 1.25.0 (#2309) 2022-08-11 23:35:34 +00:00
dependabot[bot]
046c7e5ae8 build(deps): bump github.com/aws/aws-sdk-go from 1.44.72 to 1.44.73 (#2304) 2022-08-10 23:41:39 +00:00
dependabot[bot]
8b156b19d3 build(deps): bump google.golang.org/api from 0.91.0 to 0.92.0 (#2305) 2022-08-10 23:39:48 +00:00
Jarek Kowalski
ae833bf822 refactor(repository): refactored v1 encryption overhead to be a function that's only invoked when actual V1 index is opened (#2300) 2022-08-10 05:26:51 +00:00
dependabot[bot]
37830e59d8 build(deps): bump github.com/minio/minio-go/v7 from 7.0.33 to 7.0.34 (#2273) 
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.33 to 7.0.34.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.33...v7.0.34)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-09 20:04:15 -07:00
dependabot[bot]
aac53afdf3 build(deps): bump github.com/aws/aws-sdk-go from 1.44.71 to 1.44.72 (#2297) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.71 to 1.44.72.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.71...v1.44.72)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-09 19:32:56 -07:00
Jarek Kowalski
6097e1a21c fix(ci): another attempt to fix flaky HTMLUI test (#2298) 2022-08-10 02:05:02 +00:00
Ricardo Pescuma Domenecci
0724511283 feat(repository): Error correction for blobs (#2270) 
* Initial implementation of ecc using Encryptor interface

* Created benchmark ecc command

* Fixing the order inside the wrapper

* Removed rs_bw because it is always worse

* Fixing naming and adding more comments

* Different approaches depending of file size/space overhead

* Fixes requested in PR

* Fixed lint errors

* Fixes requested in the PR

* Fixed import order

* Fixed more lint errors
 2022-08-09 18:24:25 -07:00
dependabot[bot]
3f37c86ab7 build(deps): bump github.com/prometheus/client_golang (#2296) 2022-08-09 23:28:40 +00:00
chaitalisg
b61a7627e9 test(general): Add test for snapshot fix command (#2018) 
* add robustness test for snapshot fix commands

* temporary change to see if plumbing works

* add robustness test for snapshot fix commands

* temporary change to see if plumbing works

* debug

* revert changes to robustness main_test

* Move recovery tests to a new package

* Revert unwanted changes

* Updated make target

* Temporary change for a pipeline run

* Resolve go package issues

* Remove local path used for debugging

* Debugging

* Debugging

* Add new orchestration script for recovery test

* Add comments

* Add comments

* Revert changes to robustness job script

* Remove unawanted files

* Change permissions for the orchestration script

* Fix typo

* Executable

* Update orchestration script

* Fix linter errors

* Fix linter errors

* Fix linter errors

* fix: linter grouping issue

* Fix linter errors

* Fix linter errors

* Update tests/recovery/blobmanipulator/blobmanipulator.go

Co-authored-by: Aaron Alpar <55999015+aaron-kasten@users.noreply.github.com>

* Implement feedback

* Implement feedback

* Fix linter errors

* Address feedback

* Add maintenance functions

* Added mechanism to interrupt kopia operation during the test run

* Fix linter errors

* Revert debugging changes

* Fix linter errors

* Fix linter errors

* Replace syscall

* Skip test when KOPIA_EXE is not set

* Remove debugging changes

* Add kopia command interruption to simulate corruption

* Check if command is running before killing

* Delete a snapshot before calling maintenance

* Remove nil check for the process

Co-authored-by: Shikhar Mall <small@kopia.io>
Co-authored-by: Aaron Alpar <55999015+aaron-kasten@users.noreply.github.com>
 2022-08-09 09:44:18 -07:00
Jarek Kowalski
4d55f91070 chore(ci): disable long file names in GetInterestingTempDirectoryName() when ENABLE_LONG_FILENAMES=false (#2295) 2022-08-09 06:55:30 +00:00
Jarek Kowalski
51dcaa985d chore(ci): upgraded linter to 1.48.0 (#2294) 
Mechanically fixed all issues, added `lint-fix` make target.
 2022-08-09 06:07:54 +00:00
Jarek Kowalski
419c7acb11 fix(repository): fixed V1 key derivation bug from previous refactoring (#2286) 
See 23299c3451
 2022-08-08 21:45:08 -07:00
dependabot[bot]
5fce956853 build(deps-dev): bump electron from 19.0.8 to 20.0.1 in /app (#2288) 
Bumps [electron](https://github.com/electron/electron) from 19.0.8 to 20.0.1.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v19.0.8...v20.0.1)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-08 21:24:38 -07:00
dependabot[bot]
087758556c build(deps): bump github.com/aws/aws-sdk-go from 1.44.70 to 1.44.71 (#2292) 2022-08-08 23:38:58 +00:00
dependabot[bot]
07ec981a25 build(deps-dev): bump asar from 3.1.0 to 3.2.0 in /app (#2258) 
Bumps [asar](https://github.com/electron/asar) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/electron/asar/releases)
- [Changelog](https://github.com/electron/asar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/electron/asar/compare/v3.1.0...v3.2.0)

---
updated-dependencies:
- dependency-name: asar
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-07 15:36:37 -07:00
dependabot[bot]
3c63203eee build(deps): bump electron-updater from 5.0.5 to 5.2.1 in /app (#2259) 
Bumps [electron-updater](https://github.com/electron-userland/electron-builder/tree/HEAD/packages/electron-updater) from 5.0.5 to 5.2.1.
- [Release notes](https://github.com/electron-userland/electron-builder/releases)
- [Changelog](https://github.com/electron-userland/electron-builder/blob/master/packages/electron-updater/CHANGELOG.md)
- [Commits](https://github.com/electron-userland/electron-builder/commits/electron-updater@5.2.1/packages/electron-updater)

---
updated-dependencies:
- dependency-name: electron-updater
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-07 15:36:24 -07:00
Jarek Kowalski
03159a1ff3 chore(ci): add logging to identify the cause of test flakes (#2287) 2022-08-07 09:56:14 -07:00