Commit Graph

7 Commits

Author SHA1 Message Date
Jarek Kowalski
b55d5b474c refactor(repository): refactored internal index read API to reduce memory allocations (#3754)
* refactor(repository): refactored internal index read API to reduce memory allocations

* fixed stress test flake, improved debuggability

* fixed spurious checklocks failures

* post-merge fixes

* pr feedback
2024-04-12 22:59:11 -07:00
Jarek Kowalski
09415e0c7d chore(ci): upgraded to go 1.22 (#3746)
Upgrades go to 1.22 and switches to new-style for loops

---------

Co-authored-by: Julio López <1953782+julio-lopez@users.noreply.github.com>
2024-04-08 09:52:47 -07:00
Jarek Kowalski
7ee30b76bb fix(repository): fixed handling of content.Info (#3356)
* fix(repository): fixed handling of content.Info

Previously content.Info was an interface which was implemented by:

* index.InfoStruct
* index.indexEntryInfoV1
* index.indexEntryInfoV2

The last 2 implementations were relying on memory-mapped files
which in rare cases could be closed while Kopia was still processing
them leading to #2599.

This changes fixes the bug and strictly separates content.Info (which
is now always a struct) from the other two (which were renamed as
index.InfoReader and only used inside repo/content/...).

In addition to being safer, this _should_ reduce memory allocations.

* reduce the size of content.Info with proper alignment.

* pr feedback

* renamed index.InfoStruct to index.Info
2023-10-14 10:34:15 -07:00
Jarek Kowalski
51dcaa985d chore(ci): upgraded linter to 1.48.0 (#2294)
Mechanically fixed all issues, added `lint-fix` make target.
2022-08-09 06:07:54 +00:00
Jarek Kowalski
9bf9cac7fb refactor(repository): ensure we always parse content.ID and object.ID (#1960)
* refactor(repository): ensure we always parse content.ID and object.ID

This changes the types to be incompatible with string to prevent direct
conversion to and from string.

This has the additional benefit of reducing number of memory allocations
and bytes for all IDs.

content.ID went from 2 allocations to 1:
   typical case 32 characters + 16 bytes per-string overhead
   worst-case 65 characters + 16 bytes per-string overhead
   now: 34 bytes

object.ID went from 2 allocations to 1:
   typical case 32 characters + 16 bytes per-string overhead
   worst-case 65 characters + 16 bytes per-string overhead
   now: 36 bytes

* move index.{ID,IDRange} methods to separate files

* replaced index.IDFromHash with content.IDFromHash externally

* minor tweaks and additional tests

* Update repo/content/index/id_test.go

Co-authored-by: Julio Lopez <1953782+julio-lopez@users.noreply.github.com>

* Update repo/content/index/id_test.go

Co-authored-by: Julio Lopez <1953782+julio-lopez@users.noreply.github.com>

* pr feedback

* post-merge fixes

* pr feedback

* pr feedback

* fixed subtle regression in sortedContents()

This was actually not producing invalid results because of how base36
works, just not sorting as efficiently as it could.

Co-authored-by: Julio Lopez <1953782+julio-lopez@users.noreply.github.com>
2022-05-25 14:15:56 +00:00
Jarek Kowalski
c4ab086d5f chore(repository): streamlined internal content cache API (#1828)
moved from ./repo/content to ./internal/cache
2022-03-15 23:57:04 -07:00
Jarek Kowalski
69dc7ba969 feat(repository): added 'hint' to Prefetch methods. (#1825) 2022-03-12 23:16:39 -08:00