mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-01-26 15:28:06 -05:00
Code Issues Packages Projects Releases Wiki Activity
2,049 Commits 12 Branches 88 Tags
f2b23c0400aff97a232d39c901bc845edeabec46
Commit Graph

2049 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot]
f2b23c0400 build(deps-dev): bump @testing-library/jest-dom in /htmlui (#1555) 
Bumps [@testing-library/jest-dom](https://github.com/testing-library/jest-dom) from 5.15.0 to 5.15.1.
- [Release notes](https://github.com/testing-library/jest-dom/releases)
- [Changelog](https://github.com/testing-library/jest-dom/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/jest-dom/compare/v5.15.0...v5.15.1)

---
updated-dependencies:
- dependency-name: "@testing-library/jest-dom"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 20:23:10 -08:00
dependabot[bot]
b05afc0a23 build(deps-dev): bump electron from 15.3.0 to 16.0.3 in /app (#1547) 
Bumps [electron](https://github.com/electron/electron) from 15.3.0 to 16.0.3.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v15.3.0...v16.0.3)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 20:22:38 -08:00
dependabot[bot]
66d66fac75 build(deps-dev): bump postcss from 8.3.11 to 8.4.4 in /htmlui (#1546) 
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.11 to 8.4.4.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.11...8.4.4)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 18:01:51 -08:00
dependabot[bot]
66d058b28d build(deps): bump github.com/mattn/go-colorable from 0.1.11 to 0.1.12 (#1551) 
Bumps [github.com/mattn/go-colorable](https://github.com/mattn/go-colorable) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/mattn/go-colorable/releases)
- [Commits](https://github.com/mattn/go-colorable/compare/v0.1.11...v0.1.12)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-colorable
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 18:01:35 -08:00
dependabot[bot]
75000e98ef build(deps-dev): bump concurrently from 6.3.0 to 6.4.0 in /app (#1548) 
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 6.3.0 to 6.4.0.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](https://github.com/open-cli-tools/concurrently/compare/v6.3.0...v6.4.0)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 18:00:55 -08:00
dependabot[bot]
ad52cf8266 build(deps): bump github.com/minio/minio-go/v7 from 7.0.15 to 7.0.16 (#1553) 
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.15 to 7.0.16.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.15...v7.0.16)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 18:00:42 -08:00
dependabot[bot]
d3ca671ab0 build(deps): bump react-bootstrap from 2.0.2 to 2.0.3 in /htmlui (#1556) 
Bumps [react-bootstrap](https://github.com/react-bootstrap/react-bootstrap) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/react-bootstrap/react-bootstrap/releases)
- [Changelog](https://github.com/react-bootstrap/react-bootstrap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/react-bootstrap/react-bootstrap/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: react-bootstrap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-12-01 18:00:27 -08:00
Jarek Kowalski
dc964bee43 ui: Policy Editor - show effective value and definition point for policy fields (#1545) 
* policy: resolve API for policy editor

* htmlui: enhanced Policy Editor UI to preview effective values
 2021-11-30 21:40:41 -08:00
Jarek Kowalski
93930d20cb policy: revamped policy merge mechanism (#1538) 
Added policy.Definition which allows us to precisely report where
each piece of policy came from.

Fixed a one-off bug with "noParent", which prevented merging of parent
policies one level too soon.

Added a whole bunch of merging helpers and generic reflection-based
test that ensures every single merge is tested.
 2021-11-27 18:14:45 -08:00
Lukas Rieger
5224f79d7d [snapshot restore] use non-atomic writes (#1534) 
* don't flush every file two times on snapshot restore
 2021-11-26 13:10:44 -08:00
Jarek Kowalski
b91d8ac992 ui: revamped policy editor + various UI cleanups. (#1537) 
* ui: revamped policy editor + various UI cleanups.

See: https://imgur.com/a/ghCYXIz

* htmlui: added logging policy editor
 2021-11-26 11:50:31 -08:00
CrendKing
724a26c235 Add doc for sharding (#1536) 2021-11-24 05:39:29 -08:00
Fedor Suchkov
3558d8e8b8 Replace google with gcs in examples (#1533) 
Google Cloud Storage repository creation and connection should use "gcs" instead of "google" in cli examples.
 2021-11-23 07:21:19 -08:00
Jarek Kowalski
fddc2c32c5 htmlui: upgraded react-bootstrap to 2.x (#1532) 2021-11-22 20:32:58 -08:00
Jarek Kowalski
8ab3e049d2 cli: fixed 'snapshot list --json --max-results' (#1529) 
Fixes #1455
v0.9.6 		2021-11-20 22:42:27 -08:00
Jarek Kowalski
525720db95 cli: added 'snapshot pin' command (#1528) 2021-11-20 20:53:25 -08:00
CrendKing
4bebaa7198 Cleanup cli2md to fix issue #1440 (#1525) 2021-11-20 20:33:06 -08:00
Jarek Kowalski
a5d689eb36 ui: Added test to verify #1057 (#1526) 2021-11-20 11:43:03 -08:00
CrendKing
2394b420b0 Change Mbit/s units to MB/s (base-10) (#1522) 2021-11-18 15:41:40 -08:00
Jarek Kowalski
73329a5036 ui: fixed 'Do not validate TLS certificate' caused by typo (#1523) 
* ui: fixed 'Do not validate TLS certificate' caused by typo

It was passing incorrect JSON key to the server.

Fixed by s/doNotValidateTLS/doNotVerifyTLS/g

* blob: added test for self-signed minio cert
 2021-11-18 07:48:23 -08:00
Jarek Kowalski
62edab618f throtting: implemented a Throttler based on token bucket and configur… (#1512) 
* throtting: implemented a Throttler based on token bucket and configurable window.

* cli: rewired throttle options to use common Limits structure and helpers

The JSON is backwards compatible.

* blob: remove explicit throttling from gcs,s3,b2 & azure

* cleanup: removed internal/throttle

* repo: add throttling wrapper around storage at the repository level

* throttling: expose APIs to get limits and add validation

* server: expose API to get/set throttle in a running server

* pr feedback
 2021-11-16 07:39:26 -08:00
Jarek Kowalski
cead806a3f blob: changed default shards from {3,3} to {1,3} (#1513) 
* blob: changed default shards from {3,3} to {1,3}

Turns out for very large repository around 100TB (5M blobs),
we end up creating max ~16M directories which is way too much
and slows down listing. Currently each leaf directory only has a handful
of files.

Simple sharding of {3} should work much better and will end up creating
directories with meaningful shard sizes - 12 K files per directory
should not be too slow and will reduce the overhead of listing by
4096 times.

The change is done in a backwards-compatible way and will respect
custom sharding (.shards) file written by previous 0.9 builds
as well as older repositories that don't have the .shards file (which
we assume to be {3,3}).

* fixed compat tests
 2021-11-16 06:02:04 -08:00
Shikhar Mall
2857c4831a storage api put-blob retention options (#1511) 
* storage api put-blob retention options

Co-authored-by: Shikhar Mall <shikhar@kasten.io>
 2021-11-15 19:46:42 -08:00
Jarek Kowalski
250e0dd0ab blob: added throttling wrapper (#1510) 
Throttler is only mocked for unit test purposes and not implemented yet.
 2021-11-11 18:09:18 -08:00
Jarek Kowalski
8a4ac4dec3 Upgraded linter to 1.43.0 (#1505) 
* fixed new gocritic violations
* fixed new 'contextcheck' violations
* fixed 'gosec' warnings
* suppressed ireturn and varnamelen linters
* fixed tenv violations, enabled building robustness tests on arm64
* fixed remaining linux failures
* makefile: fixed 'lint-all' target when running on arm64
* linter: increase deadline
* disable nilnil linter - to be enabled in separate PR
 2021-11-11 17:03:11 -08:00
Janne Johansson
3336a02bab Add OpenBSD instructions (#1508) 2021-11-10 08:59:53 -08:00
Jarek Kowalski
e9303cec8d upload: avoid reporting ignored entries twice (#1506) 
Previously ignored entries were reported from the estimate goroutine
and from the main upload goroutine doubling each ignored counter in
the UI.
 2021-11-09 08:39:47 -08:00
Jarek Kowalski
ca4bf81b2f ci: dependency upgrades (#1504) 
* ci: upgraded Node to 16.13.0 LTS

* ci: upgraded Hugo to 0.89.2
 2021-11-09 05:50:48 -08:00
Jarek Kowalski
b4619ffacf npm: moved build-time dependencies to devDependencies section, explicit npm audit (#1497) v0.9.5 2021-11-07 08:59:25 -08:00
Jarek Kowalski
92607a4966 maintenance: fail when the maintenance schedule cannot be read (#1499) 2021-11-07 08:59:10 -08:00
Jarek Kowalski
89edfbf257 maintenance: send logs to content log as well (#1496) 2021-11-06 23:08:00 -07:00
Jarek Kowalski
e41c53b01b server: ensure all HTTP requests are processed in a detached context (#1495) 2021-11-06 17:35:57 -07:00
Jarek Kowalski
03def8f33a server: maintenance in newly-created repo (#1494) 
The issue in #1439 was caused by goroutine context being associated
with the HTTP request so it became canceled soon after the request was
over, thus the goroutine to run maintenance never ran.

Fixed by adding ctxutil.Detach()

Also fixed logging by passing top-level contexts to requests
and added --log-server-requests flag to `server start` which enables
request logging.
 2021-11-06 17:10:53 -07:00
Jarek Kowalski
669b8515e1 epoch: eliminate redundant ListBlobs() calls in epoch.Manager (#1492) 2021-11-06 16:44:45 -07:00
Jarek Kowalski
0b737c170d maintenance: improved scheduling (#1493) 
Instead of attempting maintenance every 10 minutes we will do a longer
sleep until the predicted next maintenance time (or 4 hours, whichever
is shorter).

Related #1439
 2021-11-06 16:44:27 -07:00
Jarek Kowalski
aa40391eed performance: optimized restore performance for webdav/rclone (#1491) 
This improves the performance of partial data reads, such as the ones
during restore by avoiding reading the full blob only to discard most
of it.

The impact on restore time is dramatic:

Restoring 5.6 GB files:132921 dirs:18980 from rclone based on local
directory:

before: >2h
after: 49.45s
 2021-11-06 11:06:04 -07:00
Jarek Kowalski
dcff6c285d Added support for logging policies (#1472) 
* policy: introduced OptionalBool - refactoring

* policy: added logging policy

* testing: added support for symlinks and modtime to mockfs

* logging: exposed NullLogger instance

* upload: emit debug logs according to logging policies

* cli: logging policy support
 2021-11-06 10:06:05 -07:00
Jarek Kowalski
c086821819 testing: rclone test cleanup - pass config file 2021-11-05 12:33:25 -07:00
Jarek Kowalski
122b4b6740 testing: fixed rclone test cleanup (#1490) 
Rclone test would leave behind more and more empty directories leading
to test timeouts.

We will use `rclone` directly to do the cleanup now instead of blob
storage API.

Fixes #1487
 2021-11-05 12:19:40 -07:00
Jarek Kowalski
2a6140d82f fixed directory read race condition (#1489) 
This was introduced by a refactoring in #1361 - unlike
ioutil.ReadDir() which internally handles list/delete race and always
returns os.FileInfo, Info() on os.DirEntry can fail if a file
is deleted right after listing it.

Fixes #1486
 2021-11-05 10:18:03 -07:00
Jarek Kowalski
5931425dc4 ci: ignore unsupported architectures when publishing RPM repo 2021-11-04 19:08:39 -07:00
Pawit Pornkitprasan
6cd53648a2 docs: improve description of CAOS and CABS (#1438) 
* docs: improve description of CAOS and CABS

* improve formatting

* update encryption algorithm name
 2021-11-04 16:58:24 -07:00
Eng Zer Jun
c3f4c41591 refactor: move from ioutil.ReadDir to os.ReadDir (#1361) 
* refactor: move from ioutil.ReadDir to os.ReadDir

This commit is an addition to PR #1360. According to
`ioutil.ReadDir` documentation (https://pkg.go.dev/io/ioutil#ReadDir),
`os.ReadDir` should be preferred as it is a more efficient and correct
implementation.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* perf: optimize localfs scan performance

Reference: https://github.com/kopia/kopia/pull/1361#issuecomment-937345195
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-11-04 16:57:24 -07:00
dependabot[bot]
b1972d4103 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /app (#1484) 
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 16:56:01 -07:00
dependabot[bot]
5d4c5adae2 build(deps): bump tmpl from 1.0.4 to 1.0.5 in /htmlui (#1485) 
Bumps [tmpl](https://github.com/daaku/nodejs-tmpl) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/daaku/nodejs-tmpl/releases)
- [Commits](https://github.com/daaku/nodejs-tmpl/commits/v1.0.5)

---
updated-dependencies:
- dependency-name: tmpl
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 16:55:45 -07:00
Jarek Kowalski
e7ed4951d6 ci: also build armv7l version of KopiaUI (#1483) 
Also fixed embedded server paths for KopiaUI.

Fixes #1448
 2021-11-04 16:50:47 -07:00
CrendKing
81f66cb2db Update Actions example of Windows VSS for self-elevation details. Fix issue #1456. (#1479) 2021-11-04 09:52:47 -07:00
Jarek Kowalski
eca9228971 ci: build arm64 linux UI (#1482) 2021-11-04 08:24:08 -07:00
dependabot[bot]
6ddac0eb5c build(deps): bump postcss from 8.3.8 to 8.3.11 in /htmlui (#1460) 
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.8 to 8.3.11.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.8...8.3.11)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-03 22:16:50 -07:00
Jarek Kowalski
a0cfa2556f introduced structural debug logging and optional JSON output (#1475) 
* logging: added Logger.Debugw(message, key1, value1, ..., keyN, valueN)

This is based on ZAP and allows structural logs to be emitted.

* cli: added --json-log-console and --json-log-file flags

* logging: updated storage logging wrapper to use structural logging

* pr feedback
 2021-11-03 21:57:37 -07:00