mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-05-14 17:56:59 -04:00
Code Issues Packages Projects Releases Wiki Activity
1,960 Commits 6 Branches 89 Tags
c8b8dc9ec73bce53d32f7239e535544fab7bf4c1
Commit Graph

1960 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jarek Kowalski
c8b8dc9ec7 localfs: added ReadDir() benchmark (#1396) 
$ go test -benchmem -count 5 -benchtime 5s -run=^$ -bench "^BenchmarkReadDir.*$" ./fs/localfs

goos: darwin
goarch: amd64
pkg: github.com/kopia/kopia/fs/localfs
cpu: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
BenchmarkReadDir0-8       	  206330	     28408 ns/op	    1880 B/op	      11 allocs/op
BenchmarkReadDir0-8       	  209636	     28399 ns/op	    1928 B/op	      11 allocs/op
BenchmarkReadDir0-8       	  208366	     28569 ns/op	    1880 B/op	      11 allocs/op
BenchmarkReadDir0-8       	  203073	     28888 ns/op	    1880 B/op	      11 allocs/op
BenchmarkReadDir0-8       	  209492	     28477 ns/op	    1928 B/op	      11 allocs/op
BenchmarkReadDir1-8       	  163614	     37236 ns/op	    3736 B/op	      21 allocs/op
BenchmarkReadDir1-8       	  151528	     38796 ns/op	    3736 B/op	      21 allocs/op
BenchmarkReadDir1-8       	  161023	     38420 ns/op	    3736 B/op	      21 allocs/op
BenchmarkReadDir1-8       	  153543	     37262 ns/op	    3736 B/op	      21 allocs/op
BenchmarkReadDir1-8       	  165090	     36362 ns/op	    3736 B/op	      21 allocs/op
BenchmarkReadDir2-8       	  137521	     43338 ns/op	    4456 B/op	      30 allocs/op
BenchmarkReadDir2-8       	  138085	     43866 ns/op	    4456 B/op	      30 allocs/op
BenchmarkReadDir2-8       	  136701	     43389 ns/op	    4456 B/op	      30 allocs/op
BenchmarkReadDir2-8       	  138116	     43342 ns/op	    4456 B/op	      30 allocs/op
BenchmarkReadDir2-8       	  138170	     43379 ns/op	    4456 B/op	      30 allocs/op
BenchmarkReadDir10-8      	   62124	     96602 ns/op	   10104 B/op	      76 allocs/op
BenchmarkReadDir10-8      	   60532	     97361 ns/op	   10104 B/op	      76 allocs/op
BenchmarkReadDir10-8      	   61009	     97615 ns/op	   10104 B/op	      76 allocs/op
BenchmarkReadDir10-8      	   62072	     96550 ns/op	   10104 B/op	      76 allocs/op
BenchmarkReadDir10-8      	   62220	     95925 ns/op	   10104 B/op	      76 allocs/op
BenchmarkReadDir100-8     	    8028	    716714 ns/op	   80763 B/op	     544 allocs/op
BenchmarkReadDir100-8     	    8066	    712266 ns/op	   80755 B/op	     544 allocs/op
BenchmarkReadDir100-8     	    8047	    712203 ns/op	   80755 B/op	     544 allocs/op
BenchmarkReadDir100-8     	    8107	    711148 ns/op	   80752 B/op	     544 allocs/op
BenchmarkReadDir100-8     	    8127	    709227 ns/op	   80753 B/op	     544 allocs/op
BenchmarkReadDir1000-8    	    1558	   3861176 ns/op	  689409 B/op	    5137 allocs/op
BenchmarkReadDir1000-8    	    1566	   3848730 ns/op	  689414 B/op	    5137 allocs/op
BenchmarkReadDir1000-8    	    1516	   3838727 ns/op	  689412 B/op	    5137 allocs/op
BenchmarkReadDir1000-8    	    1550	   3833432 ns/op	  689415 B/op	    5137 allocs/op
BenchmarkReadDir1000-8    	    1557	   3966338 ns/op	  689408 B/op	    5137 allocs/op
BenchmarkReadDir10000-8   	     178	  33901454 ns/op	 7282295 B/op	   51046 allocs/op
BenchmarkReadDir10000-8   	     177	  33865198 ns/op	 7282296 B/op	   51046 allocs/op
BenchmarkReadDir10000-8   	     172	  34687834 ns/op	 7282354 B/op	   51047 allocs/op
BenchmarkReadDir10000-8   	     174	  34483220 ns/op	 7282359 B/op	   51047 allocs/op
BenchmarkReadDir10000-8   	     178	  34227937 ns/op	 7282307 B/op	   51047 allocs/op
PASS
ok  	github.com/kopia/kopia/fs/localfs	268.048s
 2021-10-16 10:14:18 -07:00
Jarek Kowalski
ff470ba67d site: mark v0.9 as stable, updated release docs (#1393) 2021-10-15 07:12:36 -07:00
Jarek Kowalski
5d9133a72b ci: switched code coverage tool to use GH action (#1391) 2021-10-14 23:51:28 -07:00
Jarek Kowalski
56cd086f92 kopia-ui: improved auto-update experience (#1390) v0.9.2 2021-10-14 23:21:07 -07:00
Jarek Kowalski
fba0956795 logging: fixed logging from parallel work queue (#1388) 
Fixes #1387
 2021-10-14 07:14:46 -07:00
Jarek Kowalski
97c4483cc9 testing: added tests for filesystem --flat option (#1384) v0.9.1 2021-10-13 21:21:55 -07:00
Jarek Kowalski
b7a4fa56bf retry: abort retry loop on context errors (#1383) 
Fixes #1377
 2021-10-13 19:10:02 -07:00
Jarek Kowalski
befaac501b rclone: pass directory shards to webdav client (#1382) 
Fixes #1378
 2021-10-13 19:09:13 -07:00
Jarek Kowalski
7ba2132b97 logging: fixed console timestamps to be local time, more nits (#1381) 2021-10-13 08:54:57 -07:00
Jarek Kowalski
51fdbf2048 nit: minor log output cleanups (#1380) 2021-10-13 06:07:18 -07:00
Jarek Kowalski
4a47bc3210 logging: switched from go-logging to zap (#1376) 
This is much more efficient in terms of memory allocations
and speeds up backup due to less GC pressure.

Fixes #1345
 2021-10-12 22:52:24 -07:00
CrendKing
6a991c180b Add doc for Amazon S3 .storageconfig (#1375) 
* Add doc for Amazon S3 .storageconfig

* Rewrite about Glacier
 2021-10-11 22:18:11 -07:00
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Jarek Kowalski
8b760b66a8 logging: added memoization of Logger instances per context (#1369) 2021-10-09 05:02:18 -07:00
Julio Lopez
6472d171d4 Skip publishing artifacts on fork repos (#1364) v0.9.0 2021-10-07 08:42:40 -07:00
Jarek Kowalski
2fa7cfb042 localfs: fixed support for snapshotting UNC path root (\\server\path) (#1362) 
Fixes #1288
Fixes #1035
Fixes #751
 2021-10-06 20:27:45 -07:00
Eng Zer Jun
73e492c9db refactor: move from io/ioutil to io and os package (#1360) 
* refactor: move from io/ioutil to io and os package

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* chore: remove //nolint:gosec for os.ReadFile

At the time of this commit, the G304 rule of gosec does not include the
`os.ReadFile` function. We remove `//nolint:gosec` temporarily until
https://github.com/securego/gosec/pull/706 is merged.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-06 08:39:10 -07:00
Jarek Kowalski
3bb5b63289 sftp: fixed performance reggression due to connection management (#1359) 
* sftp: fixed performance reggression due to connection management

The previous pooling was causing serialization of all requests, which
was too slow.

This change effectively reverts connection pooling but adds
automatic reconnection + unit tests.

* sftp: fixed unwanted retry on initial connection
 2021-10-06 07:28:42 -07:00
Jarek Kowalski
aa44b70866 mount: fixed mounting of directories w/symlinks over WebDAV (#1358) 
Directories containing symlinks could not be mounted on Windows and
were failing with 'invalid parameter'.

This change skips those entries and emits a warning to the console.

Symlinks appear to be unsupported in WebDAV at the protocol level
and servers usually follow symlinks server-side, which is not
feasible for Kopia. In the future we may be able to do more here, but
for now it's better to not show symlinks than not show anything.

Huge thanks to Joe Occhipinti for his help getting to the bottom of this
issue.
 2021-10-05 21:23:08 -07:00
Z
46cf07aa7c add simple selector for #1320 (#1355) 2021-10-04 22:41:25 -07:00
CrendKing
93b9bf15b4 Support setting AWS S3 storage class for all types of blobs (#1335) 
* Support setting AWS S3 storage class for all types of blobs

* Read .storageconfig file

* Improve loading logic

* Hide .storageconfig from ListBlobs()
v0.9.0-rc4 		2021-10-03 19:01:39 -07:00
dependabot[bot]
cae1c44c9e build(deps): bump @testing-library/react in /htmlui (#1327) 
Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 12.0.0 to 12.1.1.
- [Release notes](https://github.com/testing-library/react-testing-library/releases)
- [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/react-testing-library/compare/v12.0.0...v12.1.1)

---
updated-dependencies:
- dependency-name: "@testing-library/react"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-03 18:13:12 -07:00
CrendKing
e7297d7111 Remove unnecessary subdirectory from Windows archive (#1352) 
* Remove unnecessary subdirectory from Windows archive

* Use Scoop extract_dir instead of changing archive structure
 2021-10-03 18:10:54 -07:00
dependabot[bot]
118706cc27 build(deps): bump postcss from 8.3.6 to 8.3.8 in /htmlui (#1323) 
Bumps [postcss](https://github.com/postcss/postcss) from 8.3.6 to 8.3.8.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.3.6...8.3.8)

---
updated-dependencies:
- dependency-name: postcss
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-03 17:16:02 -07:00
Z
b105e1ecf3 add conditional to KopiaUI makefile to build arm64 and x64 versions (#1351) 2021-10-03 16:40:29 -07:00
Jarek Kowalski
0773af775d kopia-ui: always show default repository at the top of the list (#1349) 
Also disable annoying behavior where we open all repository windows
on double click.
 2021-10-03 12:04:17 -07:00
Jarek Kowalski
698d26bdbb localfs: reduce memory usage when scanning short directories (#1343) 
* localfs: reduce memory usage when scanning short directories

We read first 100 entries to determine if the directory is short
before forking to goroutines. Also reduced goroutine count, 16 was way
too aggressive.

* localfs: fixed windows-specific behavior where os.Lstat() returns different timestamps than ReadDir()
 2021-10-02 16:26:14 -07:00
dependabot[bot]
2ebb2e4c01 build(deps-dev): bump electron from 14.0.0 to 15.1.0 in /app (#1328) 
Bumps [electron](https://github.com/electron/electron) from 14.0.0 to 15.1.0.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v14.0.0...v15.1.0)

---
updated-dependencies:
- dependency-name: electron
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 12:56:08 -07:00
Jarek Kowalski
028f8a10eb go.mod tidy (#1347) 2021-10-02 12:33:59 -07:00
dependabot[bot]
0b084727a0 build(deps-dev): bump asar from 3.0.3 to 3.1.0 in /app (#1326) 
Bumps [asar](https://github.com/electron/asar) from 3.0.3 to 3.1.0.
- [Release notes](https://github.com/electron/asar/releases)
- [Changelog](https://github.com/electron/asar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/electron/asar/compare/v3.0.3...v3.1.0)

---
updated-dependencies:
- dependency-name: asar
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 11:18:41 -07:00
dependabot[bot]
2712ae1139 build(deps): bump bootstrap from 5.1.0 to 5.1.1 in /htmlui (#1325) 
Bumps [bootstrap](https://github.com/twbs/bootstrap) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/twbs/bootstrap/releases)
- [Commits](https://github.com/twbs/bootstrap/compare/v5.1.0...v5.1.1)

---
updated-dependencies:
- dependency-name: bootstrap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-02 11:18:04 -07:00
Jarek Kowalski
9ceb731388 go.mod: upgrade minor versions of dependencies (#1340) 2021-10-02 11:16:25 -07:00
CrendKing
3ce07f326e Add example of Windows shadow copy to Actions doc (#1319) 
* Add example of Windows shadow copy to Actions doc

Co-authored-by: Jarek Kowalski <jaak@jkowalski.net>
 2021-10-02 10:50:00 -07:00
Pawit Pornkitprasan
b8efb8eaea cachefs: make cache read O(1) instead of O(n) (#1341) 
* cachefs: make cache read O(1) instead of O(n)

Move the wrapping of fs.Entry before caching rather than doing
it everytime we try to read from the cache.
 2021-10-02 10:09:33 -07:00
Jarek Kowalski
6b7c03bc97 deps: upgrade to github.com/golang-jwt/jwt/v4@4.1.0 and fixed linter (#1342) 2021-10-02 10:07:46 -07:00
dependabot[bot]
5afc01181a build(deps-dev): bump concurrently from 6.2.1 to 6.2.2 in /app (#1324) 
Bumps [concurrently](https://github.com/open-cli-tools/concurrently) from 6.2.1 to 6.2.2.
- [Release notes](https://github.com/open-cli-tools/concurrently/releases)
- [Commits](https://github.com/open-cli-tools/concurrently/compare/v6.2.1...v6.2.2)

---
updated-dependencies:
- dependency-name: concurrently
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:59:17 -07:00
dependabot[bot]
6954ccdf28 build(deps): bump axios from 0.21.4 to 0.22.0 in /htmlui (#1329) 
Bumps [axios](https://github.com/axios/axios) from 0.21.4 to 0.22.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.21.4...v0.22.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:59:01 -07:00
dependabot[bot]
1984176623 build(deps): bump github.com/fatih/color from 1.12.0 to 1.13.0 (#1334) 
Bumps [github.com/fatih/color](https://github.com/fatih/color) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/fatih/color/releases)
- [Commits](https://github.com/fatih/color/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: github.com/fatih/color
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 22:58:20 -07:00
Jarek Kowalski
d60d33bda5 upload: memory usage and speed improvements (#1336) 
* upload: do not launch error group when doing non-parallel upload

* encryption: refactored code to reduce slice allocation, this uncovered a minor subtle bug in index recovery due to manual memory management

* content: fixed use-after-free bug during index recovery that started failing test after encryption memory management improvements

* gather: tweaked retention and chunk sizes

* gather: added typicalContiguousAllocator with 8MB chunks for default configuration
 2021-10-01 22:57:37 -07:00
Pawit Pornkitprasan
4c16770a8a cli: linux: read KOPIA_USE_KEYRING env (#1337) 
On Linux, Keyring is disabled by default and `--use-keyring` must
be specified on every command-line invocation, which gets quite
annoying.

This commit allows enabling keyring from environment variable
instead.
 2021-10-01 22:56:50 -07:00
cristihcd
d9c1bd0758 minor spelling and grammar fixes (#1318) 2021-09-28 15:19:02 -07:00
Jarek Kowalski
d56b93587e cli: only read first 128 MiB of provided file for compression benchmark (#1317) v0.9.0-rc3 2021-09-26 15:08:09 -07:00
Jarek Kowalski
792cc874dc repo: allow reusing of object writer buffers (#1315) 
This reduces memory consumption and speeds up backups.

1. Backing up kopia repository (3.5 GB files:133102 dirs:20074):

before: 25s, 490 MB
after: 21s, 445 MB

2. Large files (14.8 GB, 76 files)

before: 30s, 597 MB
after: 28s, 495 MB

All tests repeated 5 times for clean local filesystem repo.
 2021-09-25 14:54:31 -07:00
Julio Lopez
0629d007fc nit: remove spurious space in Makefile commands (#1311) 2021-09-22 13:44:46 -07:00
Jarek Kowalski
36027d7131 ui: fixed directory selector and browse buttons (#1310) 2021-09-21 18:51:24 -07:00
Julio Lopez
0db80a948f Fix v0.9 release notes path (#1307) 2021-09-20 17:35:30 -07:00
Jarek Kowalski
b7abeed4e5 ui: fixed showing empty result pages (#1306) 
Fixes #1149

This is a more universal fix for #1297
 2021-09-20 06:45:15 -07:00
Jarek Kowalski
1894727458 cli: fixed windows color output (#1305) 
Fixes #1287
v0.9.0-rc2 		2021-09-19 21:59:24 -07:00
Jarek Kowalski
5ab712b3e7 cli: fixed error message when trying 'repo sync-to from-config' and the config file is connected to the repository server (#1304) 2021-09-19 21:54:47 -07:00
Jarek Kowalski
e9a82c974f ui: fix pagination after toggling 'show identical snapshots' (#1302) 2021-09-19 21:18:19 -07:00