mirror/kopia
1
0
Fork 0
mirror of https://github.com/kopia/kopia.git synced 2026-03-27 02:21:59 -04:00
Code Issues Packages Projects Releases Wiki Activity
3,856 Commits 6 Branches 88 Tags
11765d696c75dccb2baefe6fcd4bef531cc6e2ff
Commit Graph

3856 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Samuel Maynard
11765d696c feat(general): reduce number of open file descriptors for indexes on Unix (#4866) 
Close file after mmap on Unix to reduce open file descriptors.

On Unix-like platforms, close the file descriptor immediately after a successful
mmap.Map of .sndx index cache files. This keeps the mapping valid until Unmap
(per POSIX semantics) and significantly reduces steady-state FD usage when many
indexes are open, helping avoid EMFILE ("too many open files").

- Split mmapOpenWithRetry into platform-specific implementations:
  - committed_content_index_disk_cache_unix.go (!windows):
    - Map RDONLY, close FD immediately.
    - Return closer that only unmaps.
  - committed_content_index_disk_cache_windows.go (windows):
    - Keep FD open until Unmap.
    - Return closer that unmaps and closes FD.
- Remove old mmapOpenWithRetry and mmap import from
  repo/content/committed_content_index_disk_cache.go.
- Add Linux-only unit test verifying FD count does not grow proportionally:
  repo/content/committed_content_index_fd_linux_test.go
  - Creates N small indexes, opens them all, checks /proc/self/fd delta stays low.

Notes:
- Behavior unchanged on Windows due to OS semantics.
- Mapping failures close the FD to avoid leaks.
- Unlink semantics remain correct; mappings stay valid until Unmap.

* remove windows only retry logic under unix
 2025-10-22 21:16:35 -07:00
Julio Lopez
b6da922e3a fix(general): address linter warnings (#4897) 2025-10-22 19:01:10 -07:00
lyndon-li
657db9706b feat(general): interfaces and structures for maintenance stats (#4890) 
Maintenance is critical for healthy of the repository.

On the other hand, Maintenance is complex, because
it runs multiple sub tasks each may generate different
results according to the maintenance policy.
The results may include deleting/combining/adding
data/metadata to the repository.

It is worthy to add more observability for these
tasks for below reasons:

It is helpful for troubleshooting. Any data change
to the repository is critical, the observability info
helps to understand what happened during the
maintenance and why that happened.

It is helpful for users to understand/predict the
repo's behavior. The repo data may be stored
in a public cloud for which costs are sensitive
to scale/duration of data stored. On the other
hand, repository has its own policy to manage
the data, so the data is not deleted until it is
safe enough according to the policy.

The observability info helps users to
understand how much data is in-use,
how much data is out of use and
when it is deleted
 2025-10-22 18:14:10 -07:00
dependabot[bot]
0613478f61 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#4892) 
Bumps the common-golang-dependencies group with 1 update: [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go).


Updates `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` from 1.6.2 to 1.6.3
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/sdk-breaking-changes-guide-migration.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/storage/azblob/v1.6.2...sdk/storage/azblob/v1.6.3)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-version: 1.6.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-20 23:39:29 +00:00
dependabot[bot]
baf41ba46e build(deps): bump github.com/klauspost/compress from 1.18.0 to 1.18.1 (#4893) 
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.18.0...v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.18.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-20 16:31:56 -07:00
yinheli
3563fdd7fd fix(cli): make --progress flag visible and use it in sync-to command (#3542) (#4877) 
* fix(cli): improve progress output control in repository sync and documentation

- Update progress flag description from "progress bar" to "progress output" for clarity
- Document progress control features in Logging, Synchronization, and Command-Line reference
- Support --no-progress flag for cleaner automation and scripting usage
 2025-10-14 19:28:54 -07:00
Julio Lopez
8465e45693 deps(go): upgrade Go to 1.24.9 (#4883) 2025-10-14 19:04:58 -07:00
dependabot[bot]
2070cd73fd build(deps): bump github.com/gofrs/flock from 0.12.1 to 0.13.0 (#4881) 
Bumps [github.com/gofrs/flock](https://github.com/gofrs/flock) from 0.12.1 to 0.13.0.
- [Release notes](https://github.com/gofrs/flock/releases)
- [Commits](https://github.com/gofrs/flock/compare/v0.12.1...v0.13.0)

---
updated-dependencies:
- dependency-name: github.com/gofrs/flock
  dependency-version: 0.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-13 20:09:31 +00:00
dependabot[bot]
52f1b4727c build(deps): bump the common-golang-dependencies group with 7 updates (#4880) 
Bumps the common-golang-dependencies group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.42.0` | `0.43.0` |
| [golang.org/x/mod](https://github.com/golang/mod) | `0.28.0` | `0.29.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.45.0` | `0.46.0` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.31.0` | `0.32.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.36.0` | `0.37.0` |
| [golang.org/x/term](https://github.com/golang/term) | `0.35.0` | `0.36.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.29.0` | `0.30.0` |


Updates `golang.org/x/crypto` from 0.42.0 to 0.43.0
- [Commits](https://github.com/golang/crypto/compare/v0.42.0...v0.43.0)

Updates `golang.org/x/mod` from 0.28.0 to 0.29.0
- [Commits](https://github.com/golang/mod/compare/v0.28.0...v0.29.0)

Updates `golang.org/x/net` from 0.45.0 to 0.46.0
- [Commits](https://github.com/golang/net/compare/v0.45.0...v0.46.0)

Updates `golang.org/x/oauth2` from 0.31.0 to 0.32.0
- [Commits](https://github.com/golang/oauth2/compare/v0.31.0...v0.32.0)

Updates `golang.org/x/sys` from 0.36.0 to 0.37.0
- [Commits](https://github.com/golang/sys/compare/v0.36.0...v0.37.0)

Updates `golang.org/x/term` from 0.35.0 to 0.36.0
- [Commits](https://github.com/golang/term/compare/v0.35.0...v0.36.0)

Updates `golang.org/x/text` from 0.29.0 to 0.30.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.29.0...v0.30.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/mod
  dependency-version: 0.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/net
  dependency-version: 0.46.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/oauth2
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/sys
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/term
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/text
  dependency-version: 0.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-13 12:48:35 -07:00
dependabot[bot]
9d65629d6a build(deps): bump github.com/chromedp/chromedp from 0.14.1 to 0.14.2 (#4874) 
Bumps [github.com/chromedp/chromedp](https://github.com/chromedp/chromedp) from 0.14.1 to 0.14.2.
- [Release notes](https://github.com/chromedp/chromedp/releases)
- [Commits](https://github.com/chromedp/chromedp/compare/v0.14.1...v0.14.2)

---
updated-dependencies:
- dependency-name: github.com/chromedp/chromedp
  dependency-version: 0.14.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-08 05:26:18 +00:00
dependabot[bot]
7c5e320c5f build(deps): bump github.com/prometheus/common (#4873) 
Bumps the telemetry-dependencies group with 1 update: [github.com/prometheus/common](https://github.com/prometheus/common).


Updates `github.com/prometheus/common` from 0.66.1 to 0.67.1
- [Release notes](https://github.com/prometheus/common/releases)
- [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/common/compare/v0.66.1...v0.67.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-version: 0.67.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: telemetry-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-08 05:25:55 +00:00
dependabot[bot]
ebaf88b937 build(deps): bump the common-golang-dependencies group with 3 updates (#4872) 
Bumps the common-golang-dependencies group with 3 updates: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go), [golang.org/x/net](https://github.com/golang/net) and [google.golang.org/api](https://github.com/googleapis/google-api-go-client).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.12.0 to 1.13.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/sdk-breaking-changes-guide-migration.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.12.0...sdk/azcore/v1.13.0)

Updates `golang.org/x/net` from 0.44.0 to 0.45.0
- [Commits](https://github.com/golang/net/compare/v0.44.0...v0.45.0)

Updates `google.golang.org/api` from 0.251.0 to 0.252.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.251.0...v0.252.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-version: 1.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/net
  dependency-version: 0.45.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/api
  dependency-version: 0.252.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-08 05:16:44 +00:00
Julio Lopez
8b0dca07a5 build(deps): upgrade Go to 1.24.8 (#4870) 2025-10-07 12:28:30 -07:00
dependabot[bot]
d8c87a3564 build(deps): bump the common-golang-dependencies group with 3 updates (#4869) 
Bumps the common-golang-dependencies group with 3 updates: [google.golang.org/api](https://github.com/googleapis/google-api-go-client), [google.golang.org/grpc](https://github.com/grpc/grpc-go) and google.golang.org/protobuf.


Updates `google.golang.org/api` from 0.250.0 to 0.251.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.250.0...v0.251.0)

Updates `google.golang.org/grpc` from 1.75.1 to 1.76.0
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.75.1...v1.76.0)

Updates `google.golang.org/protobuf` from 1.36.9 to 1.36.10

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.251.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/grpc
  dependency-version: 1.76.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/protobuf
  dependency-version: 1.36.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-06 16:46:44 -07:00
dependabot[bot]
89cd4e547d build(deps): bump the github-actions group with 2 updates (#4857) 
Bumps the github-actions group with 2 updates: [actions/dependency-review-action](https://github.com/actions/dependency-review-action) and [github/codeql-action](https://github.com/github/codeql-action).


Updates `actions/dependency-review-action` from 4.7.3 to 4.8.0
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](595b5aeba7...56339e523c)

Updates `github/codeql-action` from 3.30.2 to 3.30.5
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](d3678e237b...3599b3baa1)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-version: 4.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: github/codeql-action
  dependency-version: 3.30.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-04 02:20:54 +00:00
Julio Lopez
0765bcaa95 chore(ci): run endurance test every 6 hours (#4861) 2025-10-01 22:53:49 -07:00
Julio Lopez
c6115fa4dd test(general): expand triggers for endurance tests (#4860) 
Run endurance tests:
- on push to the test/endurance branch;
- on workflow dispatch, it allows choosing
  the target branch for the run.
 2025-10-01 19:44:01 -07:00
Julio Lopez
36508f94be test(general): fix endurance test failures (#4859) 
Prevent running "auto-maintenance" on snapshot create.

Ref:
- #4851

Context: the test fails because there are concurrent "endurance"
runners and each of these advances the clock, some of them
significantly (action{Small,Medium,Large}ClockJump), which
causes the clock skewness check to fail when
(auto-)maintenance runs.

The test maintenance action does not experience this issue
because it runs exclusively on its own, that is, other
actions have to wait until maintenance completes.
 2025-10-01 18:30:47 -07:00
dependabot[bot]
d5878a8b87 build(deps): bump ossf/scorecard-action from 2.4.2 to 2.4.3 (#4858) 
Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.4.2 to 2.4.3.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)
- [Commits](05b42c6244...4eaacf0543)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-version: 2.4.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-10-01 23:25:55 +00:00
Julio Lopez
3853ecf81f chore(general): remove unused pproflogging package (#4850) 2025-10-01 15:30:30 -07:00
dependabot[bot]
60dfb4f01c build(deps): bump the common-golang-dependencies group with 2 updates (#4855) 
Bumps the common-golang-dependencies group with 2 updates: [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) and [google.golang.org/api](https://github.com/googleapis/google-api-go-client).


Updates `cloud.google.com/go/storage` from 1.56.2 to 1.57.0
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.56.2...spanner/v1.57.0)

Updates `google.golang.org/api` from 0.249.0 to 0.250.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.249.0...v0.250.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.57.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/api
  dependency-version: 0.250.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-30 01:45:17 +00:00
Julio Lopez
6650223291 refactor(general): cleanup observabilityFlags (#4852) 
- nit: rename function to repositoryAction.
  It always calls the action with a repository
- move allocator stats functionality to observability
- rename observability functions to start/stop. They
  start and stop more than just the metrics services.
- rename field to c.enablePProfEndpoint for clarity.
- add observability run function to make it explicit
  where start and stop are called.
 2025-09-28 23:59:27 -07:00
Julio Lopez
a0a4a7ba18 refactor(cli): ensure auto-maintenance errors are propagated (#4851) 
Ensure auto-maintenance errors are propagated.
This enables sending notifications for failed "auto-maintenances".

Preserve action callback error when closing the repository fails.
 2025-09-28 23:55:15 -07:00
Jarek Kowalski
0f7253eb66 feat(general): rewrote content logs to always be JSON-based and reorganized log structure (#4822) 
This is a breaking change to users who might be using Kopia as a library.

### Log Format

```json
{"t":"<timestamp-rfc-3389-microseconds>", "span:T1":"V1", "span:T2":"V2", "n":"<source>", "m":"<message>", /*parameters*/}
```

Where each record is associated with one or more spans that describe its scope:

* `"span:client": "<hash-of-username@hostname>"`
* `"span:repo": "<random>"` - random identifier of a repository connection (from `repo.Open`)
* `"span:maintenance": "<random>"` - random identifier of a maintenance session
* `"span:upload": "<hash-of-username@host:/path>"` - uniquely identifies upload session of a given directory
* `"span:checkpoint": "<random>"` - encapsulates each checkpoint operation during Upload
* `"span:server-session": "<random>"` -single client connection to the server
* `"span:flush": "<random>"` - encapsulates each Flush session
* `"span:maintenance": "<random>"` - encapsulates each maintenance operation
* `"span:loadIndex" : "<random>"` - encapsulates index loading operation
* `"span:emr" : "<random>"` - encapsulates epoch manager refresh
* `"span:writePack": "<pack-blob-ID>"` - encapsulates pack blob preparation and writing

(plus additional minor spans for various phases of the maintenance).

Notable points:

- Used internal zero allocation JSON writer for reduced memory usage.
- renamed `--disable-internal-log` to `--disable-repository-log` (controls saving blobs to repository)
- added `--disable-content-log` (controls writing of `content-log` files)
- all storage operations are also logged in a structural way and associated with the corresponding spans.
- all content IDs are logged in a truncated format (since first N bytes that are usually enough to be unique) to improve compressibility of logs (blob IDs are frequently repeated but content IDs usually appear just once).

This format should make it possible to recreate the journey of any single content throughout pack blobs, indexes and compaction events.
 2025-09-27 17:11:13 -07:00
Julio Lopez
e1e0423b8a refactor(general): increase number of retained maintenance run info (#4846) 2025-09-25 10:50:35 -07:00
Julio Lopez
739653f57b chore(general): set packID when creating the map entry (#4843) 
Minor code cleanup
 2025-09-24 12:13:37 -07:00
Julio Lopez
6e5c8ef4e7 chore(general): minor cleanups (#4842) 
* Remove unused return value from ListIndexBlobInfos
* Unexport index.Builder.buildStable
* Remove unnecessary OneUseBuilder.BuildStable
* Remove unnecessary `BuilderCreator` interface,
  use a function type instead.
* Cleanup comment
 2025-09-24 12:11:22 -07:00
Julio Lopez
a96be32959 test(general): stop background workers when test fails (#4841) 2025-09-23 20:20:40 -07:00
Julio Lopez
3f03531303 feat(general): content-to-pack consistency checks in maintenance (#4832) 
Probabilistically perform content-to-pack consistency checks in
different maintenance phases.
 2025-09-23 19:08:40 -07:00
Julio Lopez
aaae7e8004 fix(cli): enable quick maintenance by default on repo creation (#4840) 
Avoid disabling quick maintenance on repo creation when the
epoch manager is available.
 2025-09-23 17:27:25 -07:00
Julio Lopez
1feeb00f06 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity (#4833) 
Bumps the common-golang-dependencies group with 1 update in the / directory: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.11.0 to 1.12.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/sdk-breaking-changes-guide-migration.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.11.0...sdk/azcore/v1.12.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-version: 1.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-18 22:55:21 -07:00
Julio Lopez
490fa4b5b3 feat(cli): extend stats for content verification (#4830) 
Track and report errors separately according to the type of error:
- missing pack
- truncated pack
- unreadable content.

Add a counter stat for the contents that are
read and fully verified (via `GetContent`).

Count errors grouped by pack ID using a `CountersMap`.
This allows determining the number of referenced contents
that were missing in a particular pack.

Report the counter stats via structured logging.

---

Sample output:

$ kopia content verify --progress-interval=0.5s --download-percent=100
Listing blobs...
Listed 102 blobs.
Verifying contents...
  Verified 1 contents, 0 errors, estimating...
  Verified 279 contents, 0 errors, estimating...
  Verified 512 of 624 contents (82.1%), 0 errors, remaining 0s, ETA 2025-09-17 23:03:38 PDT
Finished verifying contents
verifyCounters:	{"verifiedContents":624,"totalErrorCount":0,"contentsInMissingPacks":0,\
"contentsInTruncatedPacks":0,"unreadableContents":0,"readContents":624,\
"missingPacks":0,"truncatedPacks":0,"corruptedPacks":0}
 2025-09-17 23:36:09 -07:00
Julio Lopez
b22e97b38c remove overriding repository format via env variable (#4826) 2025-09-17 22:28:34 -07:00
Julio Lopez
fb7ecee534 refactor(general): move content verify functionality to the content package (#4827) 
Move general functionality from the `content verify` CLI
command implementation to helpers in the content package.

The primary motivation is to allow reusing the content
verification functionality during maintenance.

A separate followup change also extends content
verification to include additional stats useful for
debugging repository corruptions.

Overview of the changes:
- Relocation of the content verification functionality
  to the content package. The entry point is
  content.WriteManager.VerifyContents.
  This is primarily code movement with no functional changes.
- Addition of unit tests for the content verification functionality
  by exercising content.WriteManager.VerifyContents.
- Minor functional change: changing the logging level from
  Error to Warn for the "inner loop" error messages. This allows
  filtering out these messages if needed, while still observing the
  error message that is logged for the overall operation.
 2025-09-17 21:44:25 -07:00
Julio Lopez
516029719d fix(docs): correct terminology (#4829) 
Other minor rewording for clarity and spurious whitespace removals.
 2025-09-17 18:27:03 -07:00
Kopia Builder [bot]
b418315245 feat(ui): upgraded htmlui to the latest version (#4824) 2025-09-16 06:00:34 +00:00
dependabot[bot]
9c5fe07ac1 build(deps): bump the common-golang-dependencies group with 2 updates (#4823) 
Bumps the common-golang-dependencies group with 2 updates: [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) and [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go).


Updates `cloud.google.com/go/storage` from 1.56.1 to 1.56.2
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.56.1...storage/v1.56.2)

Updates `github.com/Azure/azure-sdk-for-go/sdk/azcore` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/sdk-breaking-changes-guide-migration.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.19.0...sdk/azcore/v1.19.1)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.56.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-version: 1.19.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-15 22:36:16 -07:00
Julio Lopez
afe9c45109 refactor(general): use testify in content_formatter_test.go (#4775) 
Use testify in content_formatter_test.go and content_manager_test.go

Refactor TestParallelWrites to run flusher in test driver Go routine,
instead of a separate Go routine. This addresses a linter error.
 2025-09-11 18:35:54 -07:00
Julio Lopez
7f6ab935a0 build(deps): bump google.golang.org/grpc (#4814) 
Bumps the common-golang-dependencies group with 1 update: [google.golang.org/grpc](https://github.com/grpc/grpc-go).


Updates `google.golang.org/grpc` from 1.75.0 to 1.75.1
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.75.0...v1.75.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-11 17:02:09 -07:00
Julio Lopez
ec563097a1 chore(ci): cleanup parameters for actions/setup-go (#4813) 
Remove `check-latest: true`. No longer needed with actions/setup-go@v6
Add category to OSSF workflow SARIF artifact
Cleanup parameters for govulncheck
 2025-09-11 16:35:28 -07:00
dependabot[bot]
99705cc1c3 build(deps): bump the github-actions group with 3 updates (#4810) 
* build(deps): bump the github-actions group with 3 updates

Bumps the github-actions group with 3 updates: [actions/setup-go](https://github.com/actions/setup-go), [github/codeql-action](https://github.com/github/codeql-action) and [actions/stale](https://github.com/actions/stale).


Updates `actions/setup-go` from 5.5.0 to 6.0.0
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](d35c59abb0...4469467582)

Updates `github/codeql-action` from 3.30.0 to 3.30.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](2d92b76c45...d3678e237b)

Updates `actions/stale` from 9.1.0 to 10.0.0
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](5bef64f19d...3a9db7e6a4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: github/codeql-action
  dependency-version: 3.30.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: actions/stale
  dependency-version: 10.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>

* use latest Go version in govulncheck

* use sarif output format

* add explicit go-version-input and disable sarif output

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: J <1953782+julio-lopez@users.noreply.github.com>
 2025-09-11 11:14:54 -07:00
Kopia Builder [bot]
238d5657f5 feat(ui): upgraded htmlui to the latest version (#4806) 2025-09-10 01:12:52 +00:00
dependabot[bot]
9460091bb9 build(deps): bump the common-golang-dependencies group with 2 updates (#4809) 
Bumps the common-golang-dependencies group with 2 updates: [golang.org/x/net](https://github.com/golang/net) and google.golang.org/protobuf.


Updates `golang.org/x/net` from 0.43.0 to 0.44.0
- [Commits](https://github.com/golang/net/compare/v0.43.0...v0.44.0)

Updates `google.golang.org/protobuf` from 1.36.8 to 1.36.9

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.44.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/protobuf
  dependency-version: 1.36.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 17:12:19 -07:00
dependabot[bot]
24d959d0cd build(deps): bump codecov/codecov-action from 5.5.0 to 5.5.1 (#4811) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.0 to 5.5.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](fdcc847654...5a1091511a)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: 5.5.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 17:10:41 -07:00
dependabot[bot]
4ccfe53f01 build(deps): bump github.com/hashicorp/cronexpr from 1.1.2 to 1.1.3 (#4803) 
Bumps [github.com/hashicorp/cronexpr](https://github.com/hashicorp/cronexpr) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/hashicorp/cronexpr/releases)
- [Commits](https://github.com/hashicorp/cronexpr/compare/v1.1.2...v1.1.3)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/cronexpr
  dependency-version: 1.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 22:30:12 +00:00
dependabot[bot]
c13d8f9398 build(deps-dev): bump electron from 36.3.2 to 36.8.1 in /app (#4800) 
Bumps [electron](https://github.com/electron/electron) from 36.3.2 to 36.8.1.
- [Release notes](https://github.com/electron/electron/releases)
- [Changelog](https://github.com/electron/electron/blob/main/docs/breaking-changes.md)
- [Commits](https://github.com/electron/electron/compare/v36.3.2...v36.8.1)

---
updated-dependencies:
- dependency-name: electron
  dependency-version: 36.8.1
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 22:17:48 +00:00
Julio Lopez
be44e7c98a chore(ci): upgrade Go toolchain to 1.24.7 (#4807) 
Addresses various security issues as well.
 2025-09-09 15:08:18 -07:00
dependabot[bot]
f5c688a60f build(deps): bump github.com/studio-b12/gowebdav from 0.10.0 to 0.11.0 (#4804) 
Bumps [github.com/studio-b12/gowebdav](https://github.com/studio-b12/gowebdav) from 0.10.0 to 0.11.0.
- [Release notes](https://github.com/studio-b12/gowebdav/releases)
- [Commits](https://github.com/studio-b12/gowebdav/compare/v0.10.0...v0.11.0)

---
updated-dependencies:
- dependency-name: github.com/studio-b12/gowebdav
  dependency-version: 0.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 14:38:46 -07:00
dependabot[bot]
d487b89c3c build(deps): bump the telemetry-dependencies group with 2 updates (#4802) 
Bumps the telemetry-dependencies group with 2 updates: [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) and [github.com/prometheus/common](https://github.com/prometheus/common).


Updates `github.com/prometheus/client_golang` from 1.23.0 to 1.23.2
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.23.0...v1.23.2)

Updates `github.com/prometheus/common` from 0.65.0 to 0.66.1
- [Release notes](https://github.com/prometheus/common/releases)
- [Changelog](https://github.com/prometheus/common/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/common/compare/v0.65.0...v0.66.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.23.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: telemetry-dependencies
- dependency-name: github.com/prometheus/common
  dependency-version: 0.66.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: telemetry-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 14:25:57 -07:00
dependabot[bot]
9de51b5e82 build(deps): bump the common-golang-dependencies group across 1 directory with 10 updates (#4805) 
Bumps the common-golang-dependencies group with 6 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.56.0` | `1.56.1` |
| [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) | `1.18.2` | `1.19.0` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.41.0` | `0.42.0` |
| [golang.org/x/mod](https://github.com/golang/mod) | `0.27.0` | `0.28.0` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.30.0` | `0.31.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.247.0` | `0.249.0` |



Updates `cloud.google.com/go/storage` from 1.56.0 to 1.56.1
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.56.0...storage/v1.56.1)

Updates `github.com/Azure/azure-sdk-for-go/sdk/azcore` from 1.18.2 to 1.19.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/sdk-breaking-changes-guide-migration.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.18.2...sdk/azcore/v1.19.0)

Updates `golang.org/x/crypto` from 0.41.0 to 0.42.0
- [Commits](https://github.com/golang/crypto/compare/v0.41.0...v0.42.0)

Updates `golang.org/x/mod` from 0.27.0 to 0.28.0
- [Commits](https://github.com/golang/mod/compare/v0.27.0...v0.28.0)

Updates `golang.org/x/oauth2` from 0.30.0 to 0.31.0
- [Commits](https://github.com/golang/oauth2/compare/v0.30.0...v0.31.0)

Updates `golang.org/x/sync` from 0.16.0 to 0.17.0
- [Commits](https://github.com/golang/sync/compare/v0.16.0...v0.17.0)

Updates `golang.org/x/sys` from 0.35.0 to 0.36.0
- [Commits](https://github.com/golang/sys/compare/v0.35.0...v0.36.0)

Updates `golang.org/x/term` from 0.34.0 to 0.35.0
- [Commits](https://github.com/golang/term/compare/v0.34.0...v0.35.0)

Updates `golang.org/x/text` from 0.28.0 to 0.29.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.28.0...v0.29.0)

Updates `google.golang.org/api` from 0.247.0 to 0.249.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.247.0...v0.249.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.56.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: common-golang-dependencies
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/crypto
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/mod
  dependency-version: 0.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/oauth2
  dependency-version: 0.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/sync
  dependency-version: 0.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/sys
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/term
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: golang.org/x/text
  dependency-version: 0.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
- dependency-name: google.golang.org/api
  dependency-version: 0.249.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: common-golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-09 10:36:59 -07:00