Commit Graph

3660 Commits

Author SHA1 Message Date
Jarek Kowalski
2a6140d82f fixed directory read race condition (#1489)
This was introduced by a refactoring in #1361 - unlike
ioutil.ReadDir() which internally handles list/delete race and always
returns os.FileInfo, Info() on os.DirEntry can fail if a file
is deleted right after listing it.

Fixes #1486
2021-11-05 10:18:03 -07:00
Jarek Kowalski
5931425dc4 ci: ignore unsupported architectures when publishing RPM repo 2021-11-04 19:08:39 -07:00
Pawit Pornkitprasan
6cd53648a2 docs: improve description of CAOS and CABS (#1438)
* docs: improve description of CAOS and CABS

* improve formatting

* update encryption algorithm name
2021-11-04 16:58:24 -07:00
Eng Zer Jun
c3f4c41591 refactor: move from ioutil.ReadDir to os.ReadDir (#1361)
* refactor: move from ioutil.ReadDir to os.ReadDir

This commit is an addition to PR #1360. According to
`ioutil.ReadDir` documentation (https://pkg.go.dev/io/ioutil#ReadDir),
`os.ReadDir` should be preferred as it is a more efficient and correct
implementation.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* perf: optimize localfs scan performance

Reference: https://github.com/kopia/kopia/pull/1361#issuecomment-937345195
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-04 16:57:24 -07:00
dependabot[bot]
b1972d4103 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /app (#1484)
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 16:56:01 -07:00
dependabot[bot]
5d4c5adae2 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /htmlui (#1485)
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 16:55:45 -07:00
Jarek Kowalski
e7ed4951d6 ci: also build armv7l version of KopiaUI (#1483)
Also fixed embedded server paths for KopiaUI.

Fixes #1448
2021-11-04 16:50:47 -07:00
CrendKing
81f66cb2db Update Actions example of Windows VSS for self-elevation details. Fix issue #1456. (#1479) 2021-11-04 09:52:47 -07:00
Jarek Kowalski
eca9228971 ci: build arm64 linux UI (#1482) 2021-11-04 08:24:08 -07:00
dependabot[bot]
6ddac0eb5c build(deps): bump postcss from 8.3.8 to 8.3.11 in /htmlui (#1460)
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.8 to 8.3.11.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.8...8.3.11)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 22:16:50 -07:00
Jarek Kowalski
a0cfa2556f introduced structural debug logging and optional JSON output (#1475)
* logging: added Logger.Debugw(message, key1, value1, ..., keyN, valueN)

This is based on ZAP and allows structural logs to be emitted.

* cli: added --json-log-console and --json-log-file flags

* logging: updated storage logging wrapper to use structural logging

* pr feedback
2021-11-03 21:57:37 -07:00
dependabot[bot]
9a1c334881 build(deps-dev): bump electron-builder from 22.11.7 to 22.13.1 in /app (#1463)
Bumps [electron-builder](https://github.com/electron-userland/electron-builder/tree/HEAD/packages/electron-builder) from 22.11.7 to 22.13.1.
- [Release notes](https://github.com/electron-userland/electron-builder/releases)
- [Changelog](https://github.com/electron-userland/electron-builder/blob/master/packages/electron-builder/CHANGELOG.md)
- [Commits](https://github.com/electron-userland/electron-builder/commits/v22.13.1/packages/electron-builder)

---
updated-dependencies:
- dependency-name: electron-builder
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 21:39:26 -07:00
dependabot[bot]
7591db5d48 build(deps): bump @fortawesome/react-fontawesome in /htmlui (#1459)
Bumps [@fortawesome/react-fontawesome](https://github.com/FortAwesome/react-fontawesome) from 0.1.15 to 0.1.16.
- [Release notes](https://github.com/FortAwesome/react-fontawesome/releases)
- [Changelog](https://github.com/FortAwesome/react-fontawesome/blob/master/CHANGELOG.md)
- [Commits](https://github.com/FortAwesome/react-fontawesome/compare/0.1.15...0.1.16)

---
updated-dependencies:
- dependency-name: "@fortawesome/react-fontawesome"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 21:38:10 -07:00
Julio Lopez
0aaa5f901f nit: use strings.Replacer (#1476) 2021-11-03 16:39:48 -07:00
dependabot[bot]
8d2b130f97 build(deps-dev): bump electron from 15.1.0 to 15.3.0 in /app (#1461)
Bumps [electron](https://github.com/electron/electron) from 15.1.0 to 15.3.0.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v15.1.0...v15.3.0)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 15:35:45 -07:00
dependabot[bot]
06bff0238d build(deps): bump bootstrap from 5.1.1 to 5.1.3 in /htmlui (#1462)
Bumps [bootstrap](https://github.com/twbs/bootstrap) from 5.1.1 to 5.1.3.
- [Release notes](https://github.com/twbs/bootstrap/releases)
- [Commits](https://github.com/twbs/bootstrap/compare/v5.1.1...v5.1.3)

---
updated-dependencies:
- dependency-name: bootstrap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 15:35:10 -07:00
dependabot[bot]
4e10562ad4 build(deps): bump axios from 0.22.0 to 0.24.0 in /htmlui (#1464)
Bumps [axios](https://github.com/axios/axios) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 12:07:29 -07:00
David Zaninovic
540910e854 Use blob.OutputBuffer in blob.Reader interface instead of internal gather.WriteBuffer (#1452)
* Use blob.OutputBuffer in blob.Reader interface instead of internal gather.WriteBuffer

* blob: remove blob.Storage.OutputBuffer methods Append() and ToByteSlice()

(cherry picked from commit 36d30b3b5f1f916e95493ca7552e6612f56624a6)

Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
2021-11-03 11:58:49 -07:00
dependabot[bot]
a162a618a0 build(deps): bump github.com/aws/aws-sdk-go from 1.40.54 to 1.41.16 (#1474)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.54 to 1.41.16.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.40.54...v1.41.16)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 07:57:57 -07:00
dependabot[bot]
4daecf3856 build(deps): bump github.com/efarrer/iothrottler from 0.0.1 to 0.0.3 (#1470)
Bumps [github.com/efarrer/iothrottler](https://github.com/efarrer/iothrottler) from 0.0.1 to 0.0.3.
- [Release notes](https://github.com/efarrer/iothrottler/releases)
- [Commits](https://github.com/efarrer/iothrottler/compare/v0.0.1...v0.0.3)

---
updated-dependencies:
- dependency-name: github.com/efarrer/iothrottler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 07:57:41 -07:00
dependabot[bot]
3a212db75e build(deps-dev): bump concurrently from 6.2.2 to 6.3.0 in /app (#1465)
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 6.2.2 to 6.3.0.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](https://github.com/open-cli-tools/concurrently/compare/v6.2.2...v6.3.0)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 07:57:23 -07:00
CrendKing
f2d21c151e Add doc for advanced compression topics (#1451) 2021-11-03 07:56:57 -07:00
dependabot[bot]
90df855be9 build(deps): bump @testing-library/react in /htmlui (#1458)
Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 12.1.1 to 12.1.2.
- [Release notes](https://github.com/testing-library/react-testing-library/releases)
- [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/react-testing-library/compare/v12.1.1...v12.1.2)

---
updated-dependencies:
- dependency-name: "@testing-library/react"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-03 07:56:41 -07:00
dependabot[bot]
66e77831f8 build(deps): bump google.golang.org/api from 0.58.0 to 0.60.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.58.0 to 0.60.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.58.0...v0.60.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-03 07:33:06 +00:00
dependabot[bot]
4d6ab35a15 build(deps): bump google.golang.org/grpc from 1.41.0 to 1.42.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.41.0...v1.42.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-03 06:45:50 +00:00
dependabot[bot]
f25c59584b build(deps): bump cloud.google.com/go/storage from 1.17.0 to 1.18.2 (#1469)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.17.0 to 1.18.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.17.0...storage/v1.18.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-02 23:02:22 -07:00
dependabot[bot]
abcc8ee74f build(deps): bump github.com/minio/minio-go/v7 from 7.0.14 to 7.0.15 (#1468)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.14 to 7.0.15.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.14...v7.0.15)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-02 23:00:48 -07:00
dependabot[bot]
05ddfb512c build(deps): bump github.com/tg123/go-htpasswd from 1.1.0 to 1.2.0 (#1466)
Bumps [github.com/tg123/go-htpasswd](https://github.com/tg123/go-htpasswd) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tg123/go-htpasswd/releases)
- [Commits](https://github.com/tg123/go-htpasswd/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: github.com/tg123/go-htpasswd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-02 22:58:38 -07:00
Jarek Kowalski
99e30355ef testing: do not run extensive provider validation testing against AWS and Wasabi, only tiny test (#1471) 2021-11-02 22:43:28 -07:00
t-rutter
b156ff7606 ci: fix Homebrew upgrade/cleanup warning (#1447) 2021-10-25 20:28:23 -07:00
Jarek Kowalski
0d0f48a7ee clock: discard monotonic clock component in clock.Now() (#1437)
The dual time measurement is described in
https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md

The fix is to discard hidden monotonic time component of time.Time
by converting to unix time and back.

Reviewed usage of clock.Now() and replaced with timetrack.StartTimer()
when measuring time.

The problem in #1402 was that passage of time was measured using
the monotonic time and not wall clock time. When the computer goes
to sleep, monotonic time is still monotonic while wall clock time makes
a leap when the computer wakes up. This is the behavior that
epoch manager (and most other compontents in Kopia) rely upon.

Fixes #1402

Co-authored-by: Julio Lopez <julio+gh@kasten.io>
v0.9.4
2021-10-22 15:35:09 -07:00
Jarek Kowalski
c8c433fb60 cli: when migrating snapshot honor destination policies (#1433)
In particular this applies compression based on destination repository
policies.

By default ignore rules are disabled during migration to preserve all
files, but that can be optionally enabled using '--apply-ignore-rules'.

Fixes #1429
2021-10-21 22:12:06 -07:00
Jarek Kowalski
39a195c6f7 cli: clarified usage of 'policy set --snapshot-time` (#1430)
Fixes #1426
2021-10-20 09:04:44 -07:00
Jarek Kowalski
76533246b4 sftp: support for password authentication (#1425)
Fixes #1373
2021-10-20 07:33:24 -07:00
Jarek Kowalski
fa2a931891 ui: fixed the refresh button in source list (#1428) 2021-10-20 06:58:37 -07:00
Jarek Kowalski
52837d12dd site: rewrote upgrade safety notice (#1424) 2021-10-19 19:24:00 -07:00
Jarek Kowalski
3c4cf33376 cli: 'repo sync-to' will compare only unique ID not the entire format (#1423)
This allows it to tolerate format changes, such as the ones made
during upgrade.

Fixes #1421
2021-10-19 09:22:57 -07:00
Robert Schindler
3e43f8e76f Migrate: pick latest instead of oldest snapshot with --latest-only (#1420)
Fix #1414
2021-10-19 08:56:35 -07:00
Jarek Kowalski
3a40cc621f epoch: do not fail on moving deletion watermark backwards (#1422)
This is a legitimate possiblity if someone uses `--safety=none` followed
by regular maintenance.

Fixes #1312
2021-10-19 08:17:58 -07:00
Jarek Kowalski
789a739a5b server: fixed next snapshot time computation (#1418)
Fixes #1405

Moved logic to SchedulingPolicy, added unit tests.
2021-10-18 22:55:11 -07:00
Jarek Kowalski
d5c619f98d epoch: additional logging to get to the bottom of #1402 (#1417) 2021-10-18 22:39:18 -07:00
Jarek Kowalski
97e05f996e cli: benchmark compressor memory usage (#1416) 2021-10-18 21:39:00 -07:00
Jarek Kowalski
e96ddc6105 upload: fix ignoring files after action redirects snapshot directory (#1409)
Fixes #1399
Fixes #1346
v0.9.3
2021-10-17 23:02:00 -07:00
Jarek Kowalski
24d8eb75da epoch: increase epoch time and cleanup safety margin (#1407)
Reduces likelihood of #1402 until we fix the root cause.
2021-10-17 22:31:56 -07:00
Jarek Kowalski
7f98c49edc epoch: when doing background compaction and cleanup work, ignore cancelation signal (#1406)
This would frequently prevent the actual compaction from being done
leading to performance degradation.
2021-10-17 20:14:25 -07:00
Jarek Kowalski
d3568a6fdb cli: added 'index inspect --parallel' flag (#1404) 2021-10-17 18:35:21 -07:00
Jarek Kowalski
64af1396f6 mount: fixed name escaping and added disambiguation (#1403)
Fixes #1400
Fixes #1050
2021-10-17 16:32:54 -07:00
Jarek Kowalski
191a51b278 ui: fixed snapshotting UNC roots (#1401)
This was caused by additional resolution of path names only done in UI,
which caused \\hostname\share to be treated as relative and resolved
against the home directory.

Fixes #1385
Fixes #1362
2021-10-17 13:25:12 -07:00
Jarek Kowalski
d9e0723fe4 actions: expose KOPIA_ACTION and KOPIA_VERSION environment variables (#1398)
Fixes #1394
2021-10-17 11:59:36 -07:00
Jarek Kowalski
5451a19296 cli: added 'snapshot delete --all-snapshots-for-source' (#1397) 2021-10-17 09:46:44 -07:00