Jarek Kowalski 0b6d76712f feat(snapshots): added free pool of localfs entries (#2115) ...

This avoids allocations of entries as large directories are traversed.

Backing up 100k small files in a single directory:

duration: current:8.1 baseline:8.6 change:-5.7 %
avg_heap_objects: current:7330688.9 baseline:7463750.6 change:-1.8 %
avg_heap_bytes: current:829248496.6 baseline:864880473.2 change:-4.1 %
avg_ram: current:159.3 baseline:159.1 change:+0.1 %
max_ram: current:283.2 baseline:285.4 change:-0.8 %
avg_cpu: current:153.1 baseline:143.6 change:+6.7 %
max_cpu: current:295.6 baseline:292.4 change:+1.1 %

Backing up Linux 5.14.8 using --parallel=4:

duration: current:6.0 baseline:6.0 change:-0.0 %
avg_heap_objects: current:5654431.1 baseline:5845078.3 change:-3.3 %
avg_heap_bytes: current:764526988.2 baseline:806833320.4 change:-5.2 %
avg_ram: current:218.2 baseline:220.1 change:-0.9 %
max_ram: current:323.5 baseline:319.5 change:+1.3 %
avg_cpu: current:144.3 baseline:145.1 change:-0.6 %
max_cpu: current:224.6 baseline:220.6 change:+1.8 %

2022-07-01 21:12:55 -07:00

..

acl

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00

apiclient

fix(security): prevent cross-site request forgery in the UI website (#1653)

2022-01-13 11:31:51 -08:00

atomicfile

ui: fixed snapshotting UNC roots (#1401)

2021-10-17 13:25:12 -07:00

auth

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

blobtesting

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

cache

chore(general): minor cleanups (#1959)

2022-05-23 15:16:25 -07:00

clock

fix(server): sleep 30m after failed maintenance (#1684)

2022-01-27 18:27:40 -08:00

completeset

refactor: extracted complete blob set functions to separate package (#1175)

2021-07-05 17:08:40 -07:00

connection

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

ctxutil

refactor(repository): better context cancelation handling (#1802)

2022-03-06 16:56:30 -08:00

diff

refactor(snapshots): Remove remaining internal uses of Readdir (#1986)

2022-06-04 06:36:25 -07:00

dirutil

feat(providers): improved sharded directory creation (#1665)

2022-01-23 14:56:35 -08:00

editor

fix(cli): fixing ignoring lines starting with '#' in 'policy edit' (#1830)

2022-03-16 08:00:50 -07:00

epoch

chore(ci): skip non-deterministic tests when computing code coverage (#1817)

2022-03-09 07:42:01 -08:00

faketime

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

fault

cache: improved test coverage (#1582)

2021-12-13 14:25:10 -08:00

freepool

feat(snapshots): added free pool of localfs entries (#2115)

2022-07-01 21:12:55 -07:00

fshasher

refactor(snapshots): Remove remaining internal uses of Readdir (#1986)

2022-06-04 06:36:25 -07:00

fusemount

refactor(snapshots): Remove remaining internal uses of Readdir (#1986)

2022-06-04 06:36:25 -07:00

gather

refactor(general): unified logging.Logger with *zap.SugaredLogger (#2090)

2022-06-26 05:11:52 +00:00

grpcapi

feat(repository): added 'hint' to Prefetch methods. (#1825)

2022-03-12 23:16:39 -08:00

hmac

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

impossible

cache: improved test coverage (#1582)

2021-12-13 14:25:10 -08:00

indextest

chore(repository): extracted content index building and parsing into repo/content/index (#1881)

2022-04-05 18:04:50 -07:00

iocopy

feat(snapshots): support restoring sparse files (#1823)

2022-03-22 19:09:50 -07:00

listcache

storage api put-blob retention options (#1511)

2021-11-15 19:46:42 -08:00

logfile

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

memtrack

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

mockfs

feat(snapshots): added fs.Entry.Close which can be used to release any resources (#2098)

2022-06-29 07:09:33 +00:00

mount

chore(general): updated linter to 1.44.0 (#1681)

2022-01-25 21:21:13 -08:00

osexec

feat(providers): ensure Ctrl-C is not passed to rclone (#1951)

2022-05-16 05:28:57 +00:00

ospath

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00

ownwrites

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

parallelwork

chore(general): updated linter to 1.44.0 (#1681)

2022-01-25 21:21:13 -08:00

passwordpersist

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00

providervalidation

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

releasable

fix(repository): fixed slow goroutine leak from indexBlobCache, added tests (#1950)

2022-05-16 01:21:30 +00:00

remoterepoapi

feat(repository): added 'hint' to Prefetch methods. (#1825)

2022-03-12 23:16:39 -08:00

repotesting

chore(general): minor cleanups (#1959)

2022-05-23 15:16:25 -07:00

retry

feat(general): Upgrade lock retries & monitoring during open & write sessions (#1758)

2022-02-24 15:00:02 -08:00

scrubber

scrubber: added unit tests (#1581)

2021-12-13 13:33:15 -08:00

server

feat(snapshots): Allow users to dynamically create entries in a directory during an upload (#1996)

2022-06-14 19:08:49 -07:00

serverapi

refactor(general): unified logging.Logger with *zap.SugaredLogger (#2090)

2022-06-26 05:11:52 +00:00

sparsefile

chore(snapshots): unify sparse and normal FS output paths (#1981)

2022-06-14 18:09:45 +00:00

stat

chore(snapshots): unify sparse and normal FS output paths (#1981)

2022-06-14 18:09:45 +00:00

stats

chore(ci): added checklocks static analyzer (#1838)

2022-03-19 22:42:59 -07:00

testlogging

refactor(general): unified logging.Logger with *zap.SugaredLogger (#2090)

2022-06-26 05:11:52 +00:00

testutil

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

timestampmeta

blob: replaced blob.Storage.SetTime() method with blob.PutOptions.SetTime (#1595)

2021-12-18 14:00:20 -08:00

timetrack

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

tlsutil

chore(general): updated linter to 1.44.0 (#1681)

2022-01-25 21:21:13 -08:00

uitask

refactor(general): unified logging.Logger with *zap.SugaredLogger (#2090)

2022-06-26 05:11:52 +00:00

units

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00

user

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00

wcmatch

chore(general): updated linter to 1.44.0 (#1681)

2022-01-25 21:21:13 -08:00

webdavmount

refactor(snapshots): Remove remaining internal uses of Readdir (#1986)

2022-06-04 06:36:25 -07:00

workshare

fix(snapshots): fixed random deadlock when Uploader results in a failure (#2020)

2022-06-07 18:37:08 -07:00

zaplogutil

build(lint): enabled gochecknoglobals and tagged existing globals (#1664)

2022-01-15 12:54:56 -08:00