mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-07-04 10:46:00 -04:00
Code Issues Packages Projects Releases Wiki Activity
2,019 Commits 6 Branches 90 Tags
89edfbf257b888d2d3fcc428b7216cdbe883d516
Commit Graph

2019 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jarek Kowalski
89edfbf257 maintenance: send logs to content log as well (#1496) 2021-11-06 23:08:00 -07:00
Jarek Kowalski
e41c53b01b server: ensure all HTTP requests are processed in a detached context (#1495) 2021-11-06 17:35:57 -07:00
Jarek Kowalski
03def8f33a server: maintenance in newly-created repo (#1494) 
The issue in #1439 was caused by goroutine context being associated
with the HTTP request so it became canceled soon after the request was
over, thus the goroutine to run maintenance never ran.

Fixed by adding ctxutil.Detach()

Also fixed logging by passing top-level contexts to requests
and added --log-server-requests flag to `server start` which enables
request logging.
 2021-11-06 17:10:53 -07:00
Jarek Kowalski
669b8515e1 epoch: eliminate redundant ListBlobs() calls in epoch.Manager (#1492) 2021-11-06 16:44:45 -07:00
Jarek Kowalski
0b737c170d maintenance: improved scheduling (#1493) 
Instead of attempting maintenance every 10 minutes we will do a longer
sleep until the predicted next maintenance time (or 4 hours, whichever
is shorter).

Related #1439
 2021-11-06 16:44:27 -07:00
Jarek Kowalski
aa40391eed performance: optimized restore performance for webdav/rclone (#1491) 
This improves the performance of partial data reads, such as the ones
during restore by avoiding reading the full blob only to discard most
of it.

The impact on restore time is dramatic:

Restoring 5.6 GB files:132921 dirs:18980 from rclone based on local
directory:

before: >2h
after: 49.45s
 2021-11-06 11:06:04 -07:00
Jarek Kowalski
dcff6c285d Added support for logging policies (#1472) 
* policy: introduced OptionalBool - refactoring

* policy: added logging policy

* testing: added support for symlinks and modtime to mockfs

* logging: exposed NullLogger instance

* upload: emit debug logs according to logging policies

* cli: logging policy support
 2021-11-06 10:06:05 -07:00
Jarek Kowalski
c086821819 testing: rclone test cleanup - pass config file 2021-11-05 12:33:25 -07:00
Jarek Kowalski
122b4b6740 testing: fixed rclone test cleanup (#1490) 
Rclone test would leave behind more and more empty directories leading
to test timeouts.

We will use `rclone` directly to do the cleanup now instead of blob
storage API.

Fixes #1487
 2021-11-05 12:19:40 -07:00
Jarek Kowalski
2a6140d82f fixed directory read race condition (#1489) 
This was introduced by a refactoring in #1361 - unlike
ioutil.ReadDir() which internally handles list/delete race and always
returns os.FileInfo, Info() on os.DirEntry can fail if a file
is deleted right after listing it.

Fixes #1486
 2021-11-05 10:18:03 -07:00
Jarek Kowalski
5931425dc4 ci: ignore unsupported architectures when publishing RPM repo 2021-11-04 19:08:39 -07:00
Pawit Pornkitprasan
6cd53648a2 docs: improve description of CAOS and CABS (#1438) 
* docs: improve description of CAOS and CABS

* improve formatting

* update encryption algorithm name
 2021-11-04 16:58:24 -07:00
Eng Zer Jun
c3f4c41591 refactor: move from ioutil.ReadDir to os.ReadDir (#1361) 
* refactor: move from ioutil.ReadDir to os.ReadDir

This commit is an addition to PR #1360. According to
`ioutil.ReadDir` documentation (https://pkg.go.dev/io/ioutil#ReadDir),
`os.ReadDir` should be preferred as it is a more efficient and correct
implementation.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* perf: optimize localfs scan performance

Reference: https://github.com/kopia/kopia/pull/1361#issuecomment-937345195
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-11-04 16:57:24 -07:00
dependabot[bot]
b1972d4103 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /app (#1484) 
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 16:56:01 -07:00
dependabot[bot]
5d4c5adae2 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /htmlui (#1485) 
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 16:55:45 -07:00
Jarek Kowalski
e7ed4951d6 ci: also build armv7l version of KopiaUI (#1483) 
Also fixed embedded server paths for KopiaUI.

Fixes #1448
 2021-11-04 16:50:47 -07:00
CrendKing
81f66cb2db Update Actions example of Windows VSS for self-elevation details. Fix issue #1456. (#1479) 2021-11-04 09:52:47 -07:00
Jarek Kowalski
eca9228971 ci: build arm64 linux UI (#1482) 2021-11-04 08:24:08 -07:00
dependabot[bot]
6ddac0eb5c build(deps): bump postcss from 8.3.8 to 8.3.11 in /htmlui (#1460) 
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.8 to 8.3.11.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.8...8.3.11)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 22:16:50 -07:00
Jarek Kowalski
a0cfa2556f introduced structural debug logging and optional JSON output (#1475) 
* logging: added Logger.Debugw(message, key1, value1, ..., keyN, valueN)

This is based on ZAP and allows structural logs to be emitted.

* cli: added --json-log-console and --json-log-file flags

* logging: updated storage logging wrapper to use structural logging

* pr feedback
 2021-11-03 21:57:37 -07:00
dependabot[bot]
9a1c334881 build(deps-dev): bump electron-builder from 22.11.7 to 22.13.1 in /app (#1463) 
Bumps [electron-builder](https://github.com/electron-userland/electron-builder/tree/HEAD/packages/electron-builder) from 22.11.7 to 22.13.1.
- [Release notes](https://github.com/electron-userland/electron-builder/releases)
- [Changelog](https://github.com/electron-userland/electron-builder/blob/master/packages/electron-builder/CHANGELOG.md)
- [Commits](https://github.com/electron-userland/electron-builder/commits/v22.13.1/packages/electron-builder)

---
updated-dependencies:
- dependency-name: electron-builder
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 21:39:26 -07:00
dependabot[bot]
7591db5d48 build(deps): bump @fortawesome/react-fontawesome in /htmlui (#1459) 
Bumps [@fortawesome/react-fontawesome](https://github.com/FortAwesome/react-fontawesome) from 0.1.15 to 0.1.16.
- [Release notes](https://github.com/FortAwesome/react-fontawesome/releases)
- [Changelog](https://github.com/FortAwesome/react-fontawesome/blob/master/CHANGELOG.md)
- [Commits](https://github.com/FortAwesome/react-fontawesome/compare/0.1.15...0.1.16)

---
updated-dependencies:
- dependency-name: "@fortawesome/react-fontawesome"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 21:38:10 -07:00
Julio Lopez
0aaa5f901f nit: use strings.Replacer (#1476) 2021-11-03 16:39:48 -07:00
dependabot[bot]
8d2b130f97 build(deps-dev): bump electron from 15.1.0 to 15.3.0 in /app (#1461) 
Bumps [electron](https://github.com/electron/electron) from 15.1.0 to 15.3.0.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v15.1.0...v15.3.0)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 15:35:45 -07:00
dependabot[bot]
06bff0238d build(deps): bump bootstrap from 5.1.1 to 5.1.3 in /htmlui (#1462) 
Bumps [bootstrap](https://github.com/twbs/bootstrap) from 5.1.1 to 5.1.3.
- [Release notes](https://github.com/twbs/bootstrap/releases)
- [Commits](https://github.com/twbs/bootstrap/compare/v5.1.1...v5.1.3)

---
updated-dependencies:
- dependency-name: bootstrap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 15:35:10 -07:00
dependabot[bot]
4e10562ad4 build(deps): bump axios from 0.22.0 to 0.24.0 in /htmlui (#1464) 
Bumps [axios](https://github.com/axios/axios) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 12:07:29 -07:00
David Zaninovic
540910e854 Use blob.OutputBuffer in blob.Reader interface instead of internal gather.WriteBuffer (#1452) 
* Use blob.OutputBuffer in blob.Reader interface instead of internal gather.WriteBuffer

* blob: remove blob.Storage.OutputBuffer methods Append() and ToByteSlice()

(cherry picked from commit 36d30b3b5f1f916e95493ca7552e6612f56624a6)

Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
 2021-11-03 11:58:49 -07:00
dependabot[bot]
a162a618a0 build(deps): bump github.com/aws/aws-sdk-go from 1.40.54 to 1.41.16 (#1474) 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.54 to 1.41.16.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.40.54...v1.41.16)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 07:57:57 -07:00
dependabot[bot]
4daecf3856 build(deps): bump github.com/efarrer/iothrottler from 0.0.1 to 0.0.3 (#1470) 
Bumps [github.com/efarrer/iothrottler](https://github.com/efarrer/iothrottler) from 0.0.1 to 0.0.3.
- [Release notes](https://github.com/efarrer/iothrottler/releases)
- [Commits](https://github.com/efarrer/iothrottler/compare/v0.0.1...v0.0.3)

---
updated-dependencies:
- dependency-name: github.com/efarrer/iothrottler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 07:57:41 -07:00
dependabot[bot]
3a212db75e build(deps-dev): bump concurrently from 6.2.2 to 6.3.0 in /app (#1465) 
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 6.2.2 to 6.3.0.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](https://github.com/open-cli-tools/concurrently/compare/v6.2.2...v6.3.0)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 07:57:23 -07:00
CrendKing
f2d21c151e Add doc for advanced compression topics (#1451) 2021-11-03 07:56:57 -07:00
dependabot[bot]
90df855be9 build(deps): bump @testing-library/react in /htmlui (#1458) 
Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 12.1.1 to 12.1.2.
- [Release notes](https://github.com/testing-library/react-testing-library/releases)
- [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/react-testing-library/compare/v12.1.1...v12.1.2)

---
updated-dependencies:
- dependency-name: "@testing-library/react"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 07:56:41 -07:00
dependabot[bot]
66e77831f8 build(deps): bump google.golang.org/api from 0.58.0 to 0.60.0 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.58.0 to 0.60.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.58.0...v0.60.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-11-03 07:33:06 +00:00
dependabot[bot]
4d6ab35a15 build(deps): bump google.golang.org/grpc from 1.41.0 to 1.42.0 
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.41.0...v1.42.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-11-03 06:45:50 +00:00
dependabot[bot]
f25c59584b build(deps): bump cloud.google.com/go/storage from 1.17.0 to 1.18.2 (#1469) 
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...storage/v1.18.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-02 23:02:22 -07:00
dependabot[bot]
abcc8ee74f build(deps): bump github.com/minio/minio-go/v7 from 7.0.14 to 7.0.15 (#1468) 
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.14 to 7.0.15.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.14...v7.0.15)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-02 23:00:48 -07:00
dependabot[bot]
05ddfb512c build(deps): bump github.com/tg123/go-htpasswd from 1.1.0 to 1.2.0 (#1466) 
Bumps [github.com/tg123/go-htpasswd](https://github.com/tg123/go-htpasswd) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tg123/go-htpasswd/releases)
- [Commits](https://github.com/tg123/go-htpasswd/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: github.com/tg123/go-htpasswd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-02 22:58:38 -07:00
Jarek Kowalski
99e30355ef testing: do not run extensive provider validation testing against AWS and Wasabi, only tiny test (#1471) 2021-11-02 22:43:28 -07:00
t-rutter
b156ff7606 ci: fix Homebrew upgrade/cleanup warning (#1447) 2021-10-25 20:28:23 -07:00
Jarek Kowalski
0d0f48a7ee clock: discard monotonic clock component in clock.Now() (#1437) 
The dual time measurement is described in
https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md

The fix is to discard hidden monotonic time component of time.Time
by converting to unix time and back.

Reviewed usage of clock.Now() and replaced with timetrack.StartTimer()
when measuring time.

The problem in #1402 was that passage of time was measured using
the monotonic time and not wall clock time. When the computer goes
to sleep, monotonic time is still monotonic while wall clock time makes
a leap when the computer wakes up. This is the behavior that
epoch manager (and most other compontents in Kopia) rely upon.

Fixes #1402

Co-authored-by: Julio Lopez <julio+gh@kasten.io>
v0.9.4 		2021-10-22 15:35:09 -07:00
Jarek Kowalski
c8c433fb60 cli: when migrating snapshot honor destination policies (#1433) 
In particular this applies compression based on destination repository
policies.

By default ignore rules are disabled during migration to preserve all
files, but that can be optionally enabled using '--apply-ignore-rules'.

Fixes #1429
 2021-10-21 22:12:06 -07:00
Jarek Kowalski
39a195c6f7 cli: clarified usage of 'policy set --snapshot-time` (#1430) 
Fixes #1426
 2021-10-20 09:04:44 -07:00
Jarek Kowalski
76533246b4 sftp: support for password authentication (#1425) 
Fixes #1373
 2021-10-20 07:33:24 -07:00
Jarek Kowalski
fa2a931891 ui: fixed the refresh button in source list (#1428) 2021-10-20 06:58:37 -07:00
Jarek Kowalski
52837d12dd site: rewrote upgrade safety notice (#1424) 2021-10-19 19:24:00 -07:00
Jarek Kowalski
3c4cf33376 cli: 'repo sync-to' will compare only unique ID not the entire format (#1423) 
This allows it to tolerate format changes, such as the ones made
during upgrade.

Fixes #1421
 2021-10-19 09:22:57 -07:00
Robert Schindler
3e43f8e76f Migrate: pick latest instead of oldest snapshot with --latest-only (#1420) 
Fix #1414
 2021-10-19 08:56:35 -07:00
Jarek Kowalski
3a40cc621f epoch: do not fail on moving deletion watermark backwards (#1422) 
This is a legitimate possiblity if someone uses `--safety=none` followed
by regular maintenance.

Fixes #1312
 2021-10-19 08:17:58 -07:00
Jarek Kowalski
789a739a5b server: fixed next snapshot time computation (#1418) 
Fixes #1405

Moved logic to SchedulingPolicy, added unit tests.
 2021-10-18 22:55:11 -07:00
Jarek Kowalski
d5c619f98d epoch: additional logging to get to the bottom of #1402 (#1417) 2021-10-18 22:39:18 -07:00