Commit Graph

84 Commits

Author SHA1 Message Date
Jeremy Edwards
9167c08183 chore(ci): restrict GH actions for provider and stress tests to upstream repository. (#1989) 2022-05-29 02:27:04 +00:00
Jarek Kowalski
ebd8f113c6 chore(ci): set golang version explicitly to 1.18 (#1952)
We were unintentionally doing that because ^1.17 was pulling 1.18.x
2022-05-16 11:55:04 -07:00
Julio Lopez
8567db6c88 build(deps): bump docker actions from 1 to 2 (#1947)
* build(deps): bump docker/setup-qemu-action from 1 to 2

Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump docker/setup-buildx-action from 1 to 2

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 00:03:35 +00:00
Julio Lopez
81dd22f8ce Dependency Review Action (#1894)
This Action will scan dependency manifest files that change as part of a Pull
Reqest, surfacing known-vulnerable versions of the packages declared or updated
in the PR. Once installed, if the workflow run is marked as required, PRs
introducing known-vulnerable packages will be blocked from merging.

Source repository: https://github.com/actions/dependency-review-action

Public documentation: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement
2022-04-19 06:57:34 +00:00
Julio Lopez
e1c74a5dab build(deps): bump actions/download-artifact from 2 to 3 (#1893)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-19 05:07:37 +00:00
Julio Lopez
4a59ce4165 build(deps): bump codecov/codecov-action from 2 to 3 (#1891)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-15 06:32:06 +00:00
Julio Lopez
6fa621941f build(deps): bump actions/upload-artifact from 2 to 3 (#1889)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-13 23:05:08 +00:00
Julio Lopez
83bc416aeb build(deps): bump actions/setup-go from 2 to 3 (#1888)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2 to 3.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-13 22:25:48 +00:00
Jarek Kowalski
766cb57160 feat(ci): automatically bump homebrew version on tag (#1807) 2022-03-06 21:44:13 -08:00
dependabot[bot]
c7ccf4c79d build(deps): bump actions/checkout from 2 to 3 (#1782)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 21:22:55 -08:00
Jarek Kowalski
a48e24e693 feat(providers): add Google Drive support (#1731)
* feat(provider): Add Google Drive support.

Co-authored-by: xkxx <xkxx@users.noreply.github.com>
Co-authored-by: xkxx <xkxiang@gmail.com>
2022-02-16 22:34:48 -08:00
Jarek Kowalski
aa6f9b17df build(ci): auto-cancel previous workflows on the same branch (#1747) 2022-02-11 18:21:11 -08:00
Jarek Kowalski
90df511609 fix(snapshots): treat empty retention policy as retaining ALL, not NONE (#1733)
This is a safety measure which addresses P0 improvement for #1732.

Given that retention policies that retain nothing make no sense, this
is not considered a breaking change.
2022-02-07 11:40:27 -08:00
Jarek Kowalski
09c1489be4 fix(ci): add rclone binaries to the list of artifacts (#1719) 2022-02-02 21:53:39 -08:00
Jarek Kowalski
f404806557 fix(ci): fixed linter issue, do not ignore in workflow (#1687) 2022-01-29 08:15:24 -08:00
Jarek Kowalski
9cad0edb53 test(ui): added end-to-end HTML UI test (#1686)
* test(general): refactored parsing of server output

* test(ui): added experimental end-to-end test using chromedp
2022-01-29 01:34:45 -08:00
Jarek Kowalski
16ffe7b5a2 fix(provider): fixed b2 provider failure (#1685)
There was a small regression in #1654 - only in negative tests.

Also removed unnecessary retries on ErrUnsupportedPutBlobOption
+ switched provider test to run once per day, instead of 12 times/day.
2022-01-28 23:19:52 -08:00
Jarek Kowalski
c2ab5020dc build(ci): auto-generate change log (#1678) 2022-01-23 15:22:11 -08:00
Jarek Kowalski
8451c622ef chore(ci): enforce pull request titles using deepakputhraya/action-pr-title (#1676) 2022-01-23 12:20:00 -08:00
Jarek Kowalski
c129f28b16 chore(ci): moved tests to standalone GHA workflow that runs in parallel (#1663) 2022-01-15 12:12:18 -08:00
Jarek Kowalski
4227de1a4b nit: fixed benign test data race (#1635)
* nit: fixed benign test data race

* run race detector tests in sequence
2021-12-30 18:20:28 -08:00
Jarek Kowalski
daacc6a8f4 htmlui: moved HTML UI to separate repository (#1600)
The source code for htmlui is now in github.com/kopia/htmlui
GitHub Actions will drop compiled builds in github.com/kopia/htmluibuild
where they are now used as a go module dependency.

This greatly simplifies the build and improves the security,
because Kopia will be consuming pre-built htmlui.

This also means kopia can now installed with embedded UI using:
`go install github.com/kopia/kopia@latest`
2021-12-18 12:28:30 -08:00
Julio Lopez
b32f2443d4 test: run provider tests on push to test/providers branch (#1594) 2021-12-16 19:45:53 -08:00
Jarek Kowalski
5d9133a72b ci: switched code coverage tool to use GH action (#1391) 2021-10-14 23:51:28 -07:00
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Julio Lopez
6472d171d4 Skip publishing artifacts on fork repos (#1364) 2021-10-07 08:42:40 -07:00
Jarek Kowalski
8b2b91f9f9 content: fixed repo upgrade version (#1286)
* content: fixed repo upgrade version

Previously upgrade would enable epoch manager and index v2 but would
not set the version of the format itself. Everything worked fine
but it would not protect from old kopia opening the repository.

* ci: added compatibility test that uses real 0.8 and current binaries
2021-09-10 22:51:51 -07:00
Julio Lopez
1077cc24d3 blob/s3: point-in-time support for s3 versioned stores (#1259)
Adds a `--point-in-time' flag for `repo connect`
2021-09-02 21:38:08 -07:00
Jarek Kowalski
cc692c40af ci: delete golang installation on armhf before installing a new one (#1276)
Per https://github.com/golang/go/issues/48134#issuecomment-911431265
2021-09-02 19:52:49 -07:00
Jarek Kowalski
06451aa7c2 upgrade golang to 1.17 + various dependencies (#1256) 2021-09-01 17:08:09 -07:00
Julio Lopez
082785af06 ci: run stress test on kopia/kopia repo (#1257) 2021-09-01 17:01:37 -07:00
Julio Lopez
310f45101f ci: avoid double workflow executions on PRs (#1258) 2021-09-01 06:06:10 -07:00
Jarek Kowalski
d77e771df7 provider test: fixed typo 2021-07-29 06:26:06 -07:00
Jarek Kowalski
7dd9e332ad ci: fixed syntax 2021-07-18 09:33:38 -07:00
Jarek Kowalski
e2e0cb37d7 ci: changed continue-on-error: true to if: succeeded() || failed() 2021-07-18 09:31:23 -07:00
Jarek Kowalski
bdd53f05d5 ci: install provider test dependencies separately 2021-07-18 09:18:14 -07:00
Jarek Kowalski
49d8a32783 testing: split provider tests into separate steps 2021-07-18 09:15:40 -07:00
Jarek Kowalski
33cf322941 ci: run provider test more often to get better stability signal 2021-07-18 09:09:43 -07:00
Jarek Kowalski
42e5661bf4 ci: added license check using github.com/frapposelli/wwhrd (#1200)
* ci: added license check using github.com/frapposelli/wwhrd

* ci: also check NPM licenses

* readme: removed FOSSA badge

* Makefile: install NPM dependencies before license check
2021-07-17 09:58:25 -07:00
Jarek Kowalski
f702e65a31 deps: removed dependency on github.com/minio/minio/pkg, replaced with command line tool (#1185) 2021-07-10 10:15:31 -07:00
Jarek Kowalski
f95ef881ce testing: fixed typo 2021-07-10 08:55:17 -07:00
Jarek Kowalski
a1aa6ae31f tests: install minio mc tool (#1184) 2021-07-10 08:53:54 -07:00
Jarek Kowalski
e64d5b8eab Fixed few subtle threading bugs uncovered by stress test and rewrote the test to be model-based (#1157)
* testing: refactored logs directory management

* content: fixed index mutex to be shared across all write sessions

added mutex protection during writecontent/refresh race

* testing: upload log artifacts

* content: bump revision number after index has been added

This fixes a bug where manifest manager in another session for
the same open repository may not see a content added, because they
will prematurely cache the incomplete set of contents.

This took 2 weeks to find.

* manifest: improved log output, fixed unnecessary mutex release

* testing: rewrote stress test to be model-based and more precise
2021-07-01 21:37:27 -07:00
Jarek Kowalski
e15a79474c azure: added support for using SAS Tokens instead of storage keys (#1093)
Fixes #1071
2021-05-22 12:43:55 -07:00
Jarek Kowalski
5179ad2cd2 cli: test + misc improvements (#1083)
* cli: Added --max-examples-per-bucket flag to 'kopia snapshot estimate'

Added and cleaned up a bunch of unit tests.

Fixes #1054

* cli: misc tests to increase code coverage of the cli package

* ci: move code coverage run into separate GH job
2021-05-17 21:47:11 -07:00
Jarek Kowalski
862298f979 testing: provider test fixes (#1073) 2021-05-15 20:50:28 -07:00
Jarek Kowalski
d81f3f1131 ci: do not upload artifacts from self-hosted runners, fixes build race (#1015)
Fixes #1014
2021-04-25 20:12:25 -07:00
Jarek Kowalski
922bc7cbc8 ci: refactored credentials handling (#987)
This strengthens credential handling after our signing keys may have
been leaked in the [codecov.io breach](https://about.codecov.io/security-update/)

* pass only minimal credentials to each build step to avoid
  exposing sensitive tokens to tools that don't need them
  (like code coverage)
* removed encrypted credential files and replaced with environment-based
* allow full ci/cd including publishing artifacts from forks
* regenerated all passwords, tokens and service accounts
* do not install Google Cloud SDK on GHA - it's already there
* moved RPM signing to 'Stage And Publish Artifacts' phase
* generated new GPG signing key

See https://kopia.discourse.group/t/important-impact-of-codecov-io-security-issue-on-kopia-build-pipeline/377
2021-04-16 08:17:13 -07:00
Jarek Kowalski
52220a75a9 testing: upload endurance test logs as artifacts regardless of status, don't ignore faulures 2021-04-07 14:21:05 -07:00
Jarek Kowalski
7c088338ce testing: upload endurance test logs as artifacts, run more frequently 2021-04-07 14:11:39 -07:00