Commit Graph

2704 Commits

Author SHA1 Message Date
Aaron H. Alpar
94ed876c73 use xor instead of fixed value 2022-11-28 12:21:47 -08:00
Aaron H. Alpar
b0aa5f67b0 fixup lint 2022-11-22 13:26:46 -08:00
Aaron H. Alpar
221976e8c5 fixup failing test 2022-11-22 12:42:59 -08:00
Aaron H. Alpar
7395aa1398 fixup named return values 2022-11-22 12:42:39 -08:00
Aaron H. Alpar
9cdae6e304 fixup for lint 2022-11-22 11:45:18 -08:00
Aaron H. Alpar
1692417831 fixup for lint 2022-11-22 11:37:32 -08:00
Aaron H. Alpar
fc39762315 fixup incorrect methods 2022-11-22 11:26:39 -08:00
Aaron H. Alpar
97f0a2b422 Merge 'master' into feat-k10-12441-B-validate-index 2022-11-22 10:14:58 -08:00
dependabot[bot]
a7d7418111 build(deps): bump github.com/minio/minio-go/v7 from 7.0.43 to 7.0.44 (#2592) 2022-11-22 06:39:45 +00:00
Jarek Kowalski
82b3f1c648 refactor(repository): refactored server repository implementations (#2595)
This extracts common bits shared by HTTP and GRPC implementations into
one.
2022-11-22 04:29:05 +00:00
Jarek Kowalski
f50779241e refactor(repository): added metrics aggregation, removed gauges (#2594) 2022-11-22 04:18:21 +00:00
dependabot[bot]
525f754a2c build(deps): bump github.com/aws/aws-sdk-go from 1.44.139 to 1.44.143 (#2593) 2022-11-21 23:51:01 +00:00
dependabot[bot]
4051630f3a build(deps): bump google.golang.org/grpc from 1.50.1 to 1.51.0 (#2591) 2022-11-21 23:38:35 +00:00
Aaron H. Alpar
7b0304131f robust commit-mode. add upgrade test 2022-11-21 10:31:18 -08:00
Aaron H. Alpar
6cc6b4a9f3 fix bug for set-parameter 2022-11-21 10:23:16 -08:00
dependabot[bot]
b5dca25d52 build(deps): bump actions/dependency-review-action from 2 to 3 (#2574)
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 2 to 3.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](https://github.com/actions/dependency-review-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-19 10:41:30 -08:00
Jarek Kowalski
65f295ed79 refactor(repository): replaced atomic values with Go 1.19 atomic wrappers (#2590)
Almost all were easy to replace, except ones exposed via JSON which
have been left as-is.

The linter has a cool behavior where it flags attempts to pass
`atomic.Int32` for example by value , which is always a mistake,
say as an argument to `fmt.Sprintf()`
2022-11-19 18:39:04 +00:00
dependabot[bot]
d0b8f5f7c0 build(deps): bump golang.org/x/crypto from 0.1.0 to 0.3.0 (#2580)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.1.0 to 0.3.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/compare/v0.1.0...v0.3.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-19 10:29:39 -08:00
dependabot[bot]
f41ecf4df8 build(deps): bump github.com/klauspost/reedsolomon from 1.11.1 to 1.11.2 (#2586)
Bumps [github.com/klauspost/reedsolomon](https://github.com/klauspost/reedsolomon) from 1.11.1 to 1.11.2.
- [Release notes](https://github.com/klauspost/reedsolomon/releases)
- [Commits](https://github.com/klauspost/reedsolomon/compare/v1.11.1...v1.11.2)

---
updated-dependencies:
- dependency-name: github.com/klauspost/reedsolomon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-19 10:29:11 -08:00
Jarek Kowalski
d782dbe550 refactor(repository): refactored Repository close logic (#2589)
This ensures metric registry is always closed on last Close().
2022-11-19 06:52:03 +00:00
Jarek Kowalski
3d38ec8405 feat(repository): added Snapshot(bool) option to reset metrics (#2588) 2022-11-19 05:29:24 +00:00
Aaron H. Alpar
2bf52cd413 fixups from PR comments 2022-11-17 09:49:35 -08:00
Aaron H. Alpar
465ef77e75 Merge 'master' into feat-k10-12441-B-validate-index 2022-11-17 08:31:54 -08:00
dependabot[bot]
6c0b01b551 build(deps): bump golang.org/x/mod from 0.6.0 to 0.7.0 (#2570)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](https://github.com/golang/mod/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-16 20:08:54 -08:00
dependabot[bot]
ce96853302 build(deps): bump github.com/aws/aws-sdk-go from 1.44.136 to 1.44.139 (#2581) 2022-11-16 23:29:28 +00:00
Aaron H. Alpar
17e8847cb5 Merge 'master' into feat-k10-12441-B-validate-index 2022-11-14 10:08:16 -08:00
dependabot[bot]
b4c8844ce1 build(deps): bump github.com/aws/aws-sdk-go from 1.44.135 to 1.44.136 (#2569) 2022-11-11 23:31:23 +00:00
dependabot[bot]
3a3e2b9512 build(deps): bump golang.org/x/net from 0.1.0 to 0.2.0 (#2566)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.1.0 to 0.2.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.1.0...v0.2.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-10 16:15:23 -08:00
dependabot[bot]
91dc0f7175 build(deps): bump github.com/aws/aws-sdk-go from 1.44.129 to 1.44.135 (#2567) 2022-11-10 23:39:25 +00:00
Aaron H. Alpar
78f9cec76a move poison blob to commit 2022-11-10 09:56:20 -08:00
Aaron H. Alpar
98cf896462 Merge 'master' into feat-k10-12441-B-validate-index 2022-11-10 09:52:57 -08:00
dependabot[bot]
3ab930a1c0 build(deps): bump minimist from 1.2.6 to 1.2.7 in /app (#2536)
Bumps [minimist](https://github.com/minimistjs/minimist) from 1.2.6 to 1.2.7.
- [Release notes](https://github.com/minimistjs/minimist/releases)
- [Changelog](https://github.com/minimistjs/minimist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/minimistjs/minimist/compare/v1.2.6...v1.2.7)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-09 22:22:21 -08:00
Jarek Kowalski
78edd92692 refactor(repository): refactored Prometheus metrics (#2532)
This may be a breaking change for users who rely on particular kopia metrics (unlikely):

- introduced blob-level metrics:

* `kopia_blob_download_full_blob_bytes_total`
* `kopia_blob_download_partial_blob_bytes_total`
* `kopia_blob_upload_bytes_total`
* `kopia_blob_storage_latency_ms` - per-method latency distribution
* `kopia_blob_errors_total` - per-method error counter

- updated cache metrics to indicate particular cache

* `kopia_cache_hit_bytes_total{cache="CACHE_TYPE"}`
* `kopia_cache_hit_total{cache="CACHE_TYPE"}`
* `kopia_cache_malformed_total{cache="CACHE_TYPE"}`
* `kopia_cache_miss_total{cache="CACHE_TYPE"}`
* `kopia_cache_miss_errors_total{cache="CACHE_TYPE"}`
* `kopia_cache_miss_bytes_total{cache="CACHE_TYPE"}`
* `kopia_cache_store_errors_total{cache="CACHE_TYPE"}`

where `CACHE_TYPE` is one of `contents`, `metadata` or `index-blobs`

- reorganized and unified content-level metrics:

* `kopia_content_write_bytes_total`
* `kopia_content_write_duration_nanos_total`

* `kopia_content_compression_attempted_bytes_total`
* `kopia_content_compression_attempted_duration_nanos_total`
* `kopia_content_compression_savings_bytes_total`
* `kopia_content_compressible_bytes_total`
* `kopia_content_non_compressible_bytes_total`
* `kopia_content_after_compression_bytes_total`

* `kopia_content_decompressed_bytes_total`
* `kopia_content_decompressed_duration_nanos_total`

* `kopia_content_encrypted_bytes_total`
* `kopia_content_encrypted_duration_nanos_total`

* `kopia_content_hashed_bytes_total`
* `kopia_content_hashed_duration_nanos_total`

* `kopia_content_deduplicated_bytes_total`

* `kopia_content_read_bytes_total`
* `kopia_content_read_duration_nanos_total`

* `kopia_content_decrypted_bytes_total`
* `kopia_content_decrypted_duration_nanos_total`

* `kopia_content_uploaded_bytes_total`

Also introduced `internal/metrics` framework which constructs Prometheus metrics in a uniform way and will allow us to include some of these metrics in telemetry report in future PRs.
2022-11-10 05:30:06 +00:00
Aaron H. Alpar
182a0ed0f1 Merge 'master' into feat-k10-12441-B-validate-index 2022-11-09 09:32:41 -08:00
dependabot[bot]
3ff7b5a840 build(deps): bump go.opentelemetry.io/otel/exporters/jaeger (#2557) 2022-11-09 00:05:31 +00:00
Aaron H. Alpar
16a517eb05 set-parameter fixup 2022-11-08 15:52:39 -08:00
dependabot[bot]
abe25d0a4d build(deps): bump google.golang.org/api from 0.102.0 to 0.103.0 (#2560) 2022-11-08 23:26:46 +00:00
dependabot[bot]
95b5f77175 build(deps): bump github.com/prometheus/client_golang (#2559) 2022-11-08 23:21:20 +00:00
Aaron H. Alpar
047c39a558 add validation to command line 2022-11-08 10:32:06 -08:00
Aaron H. Alpar
a60acd3e30 fixup bad merge 2022-11-08 10:31:30 -08:00
Aaron Alpar
636b431127 Update cli/command_repository_upgrade.go
Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
2022-11-08 10:31:30 -08:00
Aaron H. Alpar
fc37f1f1b9 add comments. fixup. 2022-11-08 10:31:30 -08:00
Aaron H. Alpar
44f5410219 add command stub 2022-11-08 10:31:30 -08:00
Aaron H. Alpar
041f04d063 add test 2022-11-08 10:31:30 -08:00
Aaron H. Alpar
8c159e0e17 add index validation 2022-11-08 10:31:30 -08:00
Aaron H. Alpar
3389a5cb6e remove unused context param 2022-11-08 10:31:29 -08:00
dependabot[bot]
243dd1c2b6 build(deps): bump cloud.google.com/go/storage from 1.27.0 to 1.28.0 (#2556) 2022-11-07 23:36:56 +00:00
ashmrtn
4024071fc0 fix(repository): Manifest label comparison when value is empty (#2543)
Allow matching on labels with empty values. Previously, the "empty
value" for a map string ("") would match the empty value of the label
key, causing the system to return all manifests. This separates key
existence from value equality to handle labels with empty values.
2022-11-04 08:02:13 -07:00
dependabot[bot]
cd4fa5dd34 build(deps): bump github.com/klauspost/compress from 1.15.11 to 1.15.12 (#2547)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.15.11 to 1.15.12.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.15.11...v1.15.12)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-04 08:01:26 -07:00
dependabot[bot]
98d2c42e52 build(deps): bump github.com/aws/aws-sdk-go from 1.44.128 to 1.44.129 (#2546) 2022-11-02 23:49:34 +00:00