mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-02-23 01:54:33 -05:00
Code Issues Packages Projects Releases Wiki Activity
1,954 Commits 5 Branches 88 Tags
b7a4fa56bf3ec7fbb42bbd89fd3f0b33a2ff6d5a
Commit Graph

1954 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jarek Kowalski
b7a4fa56bf retry: abort retry loop on context errors (#1383) 
Fixes #1377
 2021-10-13 19:10:02 -07:00
Jarek Kowalski
befaac501b rclone: pass directory shards to webdav client (#1382) 
Fixes #1378
 2021-10-13 19:09:13 -07:00
Jarek Kowalski
7ba2132b97 logging: fixed console timestamps to be local time, more nits (#1381) 2021-10-13 08:54:57 -07:00
Jarek Kowalski
51fdbf2048 nit: minor log output cleanups (#1380) 2021-10-13 06:07:18 -07:00
Jarek Kowalski
4a47bc3210 logging: switched from go-logging to zap (#1376) 
This is much more efficient in terms of memory allocations
and speeds up backup due to less GC pressure.

Fixes #1345
 2021-10-12 22:52:24 -07:00
CrendKing
6a991c180b Add doc for Amazon S3 .storageconfig (#1375) 
* Add doc for Amazon S3 .storageconfig

* Rewrite about Glacier
 2021-10-11 22:18:11 -07:00
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Jarek Kowalski
8b760b66a8 logging: added memoization of Logger instances per context (#1369) 2021-10-09 05:02:18 -07:00
Julio Lopez
6472d171d4 Skip publishing artifacts on fork repos (#1364) v0.9.0 2021-10-07 08:42:40 -07:00
Jarek Kowalski
2fa7cfb042 localfs: fixed support for snapshotting UNC path root (\\server\path) (#1362) 
Fixes #1288
Fixes #1035
Fixes #751
 2021-10-06 20:27:45 -07:00
Eng Zer Jun
73e492c9db refactor: move from io/ioutil to io and os package (#1360) 
* refactor: move from io/ioutil to io and os package

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* chore: remove //nolint:gosec for os.ReadFile

At the time of this commit, the G304 rule of gosec does not include the
`os.ReadFile` function. We remove `//nolint:gosec` temporarily until
https://github.com/securego/gosec/pull/706 is merged.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-06 08:39:10 -07:00
Jarek Kowalski
3bb5b63289 sftp: fixed performance reggression due to connection management (#1359) 
* sftp: fixed performance reggression due to connection management

The previous pooling was causing serialization of all requests, which
was too slow.

This change effectively reverts connection pooling but adds
automatic reconnection + unit tests.

* sftp: fixed unwanted retry on initial connection
 2021-10-06 07:28:42 -07:00
Jarek Kowalski
aa44b70866 mount: fixed mounting of directories w/symlinks over WebDAV (#1358) 
Directories containing symlinks could not be mounted on Windows and
were failing with 'invalid parameter'.

This change skips those entries and emits a warning to the console.

Symlinks appear to be unsupported in WebDAV at the protocol level
and servers usually follow symlinks server-side, which is not
feasible for Kopia. In the future we may be able to do more here, but
for now it's better to not show symlinks than not show anything.

Huge thanks to Joe Occhipinti for his help getting to the bottom of this
issue.
 2021-10-05 21:23:08 -07:00
Z
46cf07aa7c add simple selector for #1320 (#1355) 2021-10-04 22:41:25 -07:00
CrendKing
93b9bf15b4 Support setting AWS S3 storage class for all types of blobs (#1335) 
* Support setting AWS S3 storage class for all types of blobs

* Read .storageconfig file

* Improve loading logic

* Hide .storageconfig from ListBlobs()
v0.9.0-rc4 		2021-10-03 19:01:39 -07:00
dependabot[bot]
cae1c44c9e build(deps): bump @testing-library/react in /htmlui (#1327) 
Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 12.0.0 to 12.1.1.
- [Release notes](https://github.com/testing-library/react-testing-library/releases)
- [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/react-testing-library/compare/v12.0.0...v12.1.1)

---
updated-dependencies:
- dependency-name: "@testing-library/react"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-03 18:13:12 -07:00
CrendKing
e7297d7111 Remove unnecessary subdirectory from Windows archive (#1352) 
* Remove unnecessary subdirectory from Windows archive

* Use Scoop extract_dir instead of changing archive structure
 2021-10-03 18:10:54 -07:00
dependabot[bot]
118706cc27 build(deps): bump postcss from 8.3.6 to 8.3.8 in /htmlui (#1323) 
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.6 to 8.3.8.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.6...8.3.8)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-03 17:16:02 -07:00
Z
b105e1ecf3 add conditional to KopiaUI makefile to build arm64 and x64 versions (#1351) 2021-10-03 16:40:29 -07:00
Jarek Kowalski
0773af775d kopia-ui: always show default repository at the top of the list (#1349) 
Also disable annoying behavior where we open all repository windows
on double click.
 2021-10-03 12:04:17 -07:00
Jarek Kowalski
698d26bdbb localfs: reduce memory usage when scanning short directories (#1343) 
* localfs: reduce memory usage when scanning short directories

We read first 100 entries to determine if the directory is short
before forking to goroutines. Also reduced goroutine count, 16 was way
too aggressive.

* localfs: fixed windows-specific behavior where os.Lstat() returns different timestamps than ReadDir()
 2021-10-02 16:26:14 -07:00
dependabot[bot]
2ebb2e4c01 build(deps-dev): bump electron from 14.0.0 to 15.1.0 in /app (#1328) 
Bumps [electron](https://github.com/electron/electron) from 14.0.0 to 15.1.0.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v14.0.0...v15.1.0)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 12:56:08 -07:00
Jarek Kowalski
028f8a10eb go.mod tidy (#1347) 2021-10-02 12:33:59 -07:00
dependabot[bot]
0b084727a0 build(deps-dev): bump asar from 3.0.3 to 3.1.0 in /app (#1326) 
Bumps [asar](https://github.com/electron/asar) from 3.0.3 to 3.1.0.
- [Release notes](https://github.com/electron/asar/releases)
- [Changelog](https://github.com/electron/asar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/electron/asar/compare/v3.0.3...v3.1.0)

---
updated-dependencies:
- dependency-name: asar
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 11:18:41 -07:00
dependabot[bot]
2712ae1139 build(deps): bump bootstrap from 5.1.0 to 5.1.1 in /htmlui (#1325) 
Bumps [bootstrap](https://github.com/twbs/bootstrap) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/twbs/bootstrap/releases)
- [Commits](https://github.com/twbs/bootstrap/compare/v5.1.0...v5.1.1)

---
updated-dependencies:
- dependency-name: bootstrap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 11:18:04 -07:00
Jarek Kowalski
9ceb731388 go.mod: upgrade minor versions of dependencies (#1340) 2021-10-02 11:16:25 -07:00
CrendKing
3ce07f326e Add example of Windows shadow copy to Actions doc (#1319) 
* Add example of Windows shadow copy to Actions doc

Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
 2021-10-02 10:50:00 -07:00
Pawit Pornkitprasan
b8efb8eaea cachefs: make cache read O(1) instead of O(n) (#1341) 
* cachefs: make cache read O(1) instead of O(n)

Move the wrapping of fs.Entry before caching rather than doing
it everytime we try to read from the cache.
 2021-10-02 10:09:33 -07:00
Jarek Kowalski
6b7c03bc97 deps: upgrade to github.com/golang-jwt/jwt/v4@4.1.0 and fixed linter (#1342) 2021-10-02 10:07:46 -07:00
dependabot[bot]
5afc01181a build(deps-dev): bump concurrently from 6.2.1 to 6.2.2 in /app (#1324) 
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](https://github.com/open-cli-tools/concurrently/compare/v6.2.1...v6.2.2)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:59:17 -07:00
dependabot[bot]
6954ccdf28 build(deps): bump axios from 0.21.4 to 0.22.0 in /htmlui (#1329) 
Bumps [axios](https://github.com/axios/axios) from 0.21.4 to 0.22.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.21.4...v0.22.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:59:01 -07:00
dependabot[bot]
1984176623 build(deps): bump github.com/fatih/color from 1.12.0 to 1.13.0 (#1334) 
Bumps [github.com/fatih/color](https://github.com/fatih/color) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/fatih/color/releases)
- [Commits](https://github.com/fatih/color/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/fatih/color
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:58:20 -07:00
Jarek Kowalski
d60d33bda5 upload: memory usage and speed improvements (#1336) 
* upload: do not launch error group when doing non-parallel upload

* encryption: refactored code to reduce slice allocation, this uncovered a minor subtle bug in index recovery due to manual memory management

* content: fixed use-after-free bug during index recovery that started failing test after encryption memory management improvements

* gather: tweaked retention and chunk sizes

* gather: added typicalContiguousAllocator with 8MB chunks for default configuration
 2021-10-01 22:57:37 -07:00
Pawit Pornkitprasan
4c16770a8a cli: linux: read KOPIA_USE_KEYRING env (#1337) 
On Linux, Keyring is disabled by default and `--use-keyring` must
be specified on every command-line invocation, which gets quite
annoying.

This commit allows enabling keyring from environment variable
instead.
 2021-10-01 22:56:50 -07:00
cristihcd
d9c1bd0758 minor spelling and grammar fixes (#1318) 2021-09-28 15:19:02 -07:00
Jarek Kowalski
d56b93587e cli: only read first 128 MiB of provided file for compression benchmark (#1317) v0.9.0-rc3 2021-09-26 15:08:09 -07:00
Jarek Kowalski
792cc874dc repo: allow reusing of object writer buffers (#1315) 
This reduces memory consumption and speeds up backups.

1. Backing up kopia repository (3.5 GB files:133102 dirs:20074):

before: 25s, 490 MB
after: 21s, 445 MB

2. Large files (14.8 GB, 76 files)

before: 30s, 597 MB
after: 28s, 495 MB

All tests repeated 5 times for clean local filesystem repo.
 2021-09-25 14:54:31 -07:00
Julio Lopez
0629d007fc nit: remove spurious space in Makefile commands (#1311) 2021-09-22 13:44:46 -07:00
Jarek Kowalski
36027d7131 ui: fixed directory selector and browse buttons (#1310) 2021-09-21 18:51:24 -07:00
Julio Lopez
0db80a948f Fix v0.9 release notes path (#1307) 2021-09-20 17:35:30 -07:00
Jarek Kowalski
b7abeed4e5 ui: fixed showing empty result pages (#1306) 
Fixes #1149

This is a more universal fix for #1297
 2021-09-20 06:45:15 -07:00
Jarek Kowalski
1894727458 cli: fixed windows color output (#1305) 
Fixes #1287
v0.9.0-rc2 		2021-09-19 21:59:24 -07:00
Jarek Kowalski
5ab712b3e7 cli: fixed error message when trying 'repo sync-to from-config' and the config file is connected to the repository server (#1304) 2021-09-19 21:54:47 -07:00
Jarek Kowalski
e9a82c974f ui: fix pagination after toggling 'show identical snapshots' (#1302) 2021-09-19 21:18:19 -07:00
Jarek Kowalski
1566b74263 blob: support for custom blob store sharding (#1299) 
* blob: support for custom blob store sharding

This is experimental.

The .shards file can reside in the root of any blob storage that uses
sharding (filesystem/sftp/webdav) and can specify rules for sharding.

{
  "default": [3,2,1],
  "overrides": [
    { "prefix": "p", "shards": [2,2] },
    { "prefix": "x", "shards": [1,1,1] }
  ],
  "maxNonShardedLength": 2
}

With this in place we'll be later able to do resharding of the
repository to optimize get/put/list performance for both repositories
and caches.

* cli: command line tools to manipulate shards in a directory
 2021-09-19 18:50:38 -07:00
Jarek Kowalski
6d40d2b7ad ui: Added UI options to select shallow restore options (#1301) 2021-09-19 09:57:26 -07:00
Jarek Kowalski
d76cbcf47e cache: when cache validation fails, properly reset output buffer (#1295) 2021-09-16 05:35:54 -07:00
Jarek Kowalski
c48f2e5528 cli: added support for passing bare filename for --config-file parameter (#1293) 
When --`config-file` is passed as a filename without any directory
(absolute or relative) it is resolved in OS-specific
config path.

For example on macOS:

`--config-file foo.config`

resolves to:

`~/Library/Application Support/kopia/foo.config`
 2021-09-15 18:30:38 -07:00
Jarek Kowalski
928150fe6b linter: upgrade to 1.42.1 (#1292) 2021-09-14 19:11:39 -07:00
Jarek Kowalski
bd7e9641da fixed minor buffer leak from internal logger, added allocation tracker (#1290) 2021-09-14 06:13:19 -07:00