Commit Graph

75 Commits

Author SHA1 Message Date
dependabot[bot]
c7ccf4c79d build(deps): bump actions/checkout from 2 to 3 (#1782)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 21:22:55 -08:00
Jarek Kowalski
a48e24e693 feat(providers): add Google Drive support (#1731)
* feat(provider): Add Google Drive support.

Co-authored-by: xkxx <xkxx@users.noreply.github.com>
Co-authored-by: xkxx <xkxiang@gmail.com>
2022-02-16 22:34:48 -08:00
Jarek Kowalski
aa6f9b17df build(ci): auto-cancel previous workflows on the same branch (#1747) 2022-02-11 18:21:11 -08:00
Jarek Kowalski
90df511609 fix(snapshots): treat empty retention policy as retaining ALL, not NONE (#1733)
This is a safety measure which addresses P0 improvement for #1732.

Given that retention policies that retain nothing make no sense, this
is not considered a breaking change.
2022-02-07 11:40:27 -08:00
Jarek Kowalski
09c1489be4 fix(ci): add rclone binaries to the list of artifacts (#1719) 2022-02-02 21:53:39 -08:00
Jarek Kowalski
f404806557 fix(ci): fixed linter issue, do not ignore in workflow (#1687) 2022-01-29 08:15:24 -08:00
Jarek Kowalski
9cad0edb53 test(ui): added end-to-end HTML UI test (#1686)
* test(general): refactored parsing of server output

* test(ui): added experimental end-to-end test using chromedp
2022-01-29 01:34:45 -08:00
Jarek Kowalski
16ffe7b5a2 fix(provider): fixed b2 provider failure (#1685)
There was a small regression in #1654 - only in negative tests.

Also removed unnecessary retries on ErrUnsupportedPutBlobOption
+ switched provider test to run once per day, instead of 12 times/day.
2022-01-28 23:19:52 -08:00
Jarek Kowalski
c2ab5020dc build(ci): auto-generate change log (#1678) 2022-01-23 15:22:11 -08:00
Jarek Kowalski
8451c622ef chore(ci): enforce pull request titles using deepakputhraya/action-pr-title (#1676) 2022-01-23 12:20:00 -08:00
Jarek Kowalski
c129f28b16 chore(ci): moved tests to standalone GHA workflow that runs in parallel (#1663) 2022-01-15 12:12:18 -08:00
Jarek Kowalski
4227de1a4b nit: fixed benign test data race (#1635)
* nit: fixed benign test data race

* run race detector tests in sequence
2021-12-30 18:20:28 -08:00
Jarek Kowalski
daacc6a8f4 htmlui: moved HTML UI to separate repository (#1600)
The source code for htmlui is now in github.com/kopia/htmlui
GitHub Actions will drop compiled builds in github.com/kopia/htmluibuild
where they are now used as a go module dependency.

This greatly simplifies the build and improves the security,
because Kopia will be consuming pre-built htmlui.

This also means kopia can now installed with embedded UI using:
`go install github.com/kopia/kopia@latest`
2021-12-18 12:28:30 -08:00
Julio Lopez
b32f2443d4 test: run provider tests on push to test/providers branch (#1594) 2021-12-16 19:45:53 -08:00
Jarek Kowalski
5d9133a72b ci: switched code coverage tool to use GH action (#1391) 2021-10-14 23:51:28 -07:00
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Julio Lopez
6472d171d4 Skip publishing artifacts on fork repos (#1364) 2021-10-07 08:42:40 -07:00
Jarek Kowalski
8b2b91f9f9 content: fixed repo upgrade version (#1286)
* content: fixed repo upgrade version

Previously upgrade would enable epoch manager and index v2 but would
not set the version of the format itself. Everything worked fine
but it would not protect from old kopia opening the repository.

* ci: added compatibility test that uses real 0.8 and current binaries
2021-09-10 22:51:51 -07:00
Julio Lopez
1077cc24d3 blob/s3: point-in-time support for s3 versioned stores (#1259)
Adds a `--point-in-time' flag for `repo connect`
2021-09-02 21:38:08 -07:00
Jarek Kowalski
cc692c40af ci: delete golang installation on armhf before installing a new one (#1276)
Per https://github.com/golang/go/issues/48134#issuecomment-911431265
2021-09-02 19:52:49 -07:00
Jarek Kowalski
06451aa7c2 upgrade golang to 1.17 + various dependencies (#1256) 2021-09-01 17:08:09 -07:00
Julio Lopez
082785af06 ci: run stress test on kopia/kopia repo (#1257) 2021-09-01 17:01:37 -07:00
Julio Lopez
310f45101f ci: avoid double workflow executions on PRs (#1258) 2021-09-01 06:06:10 -07:00
Jarek Kowalski
d77e771df7 provider test: fixed typo 2021-07-29 06:26:06 -07:00
Jarek Kowalski
7dd9e332ad ci: fixed syntax 2021-07-18 09:33:38 -07:00
Jarek Kowalski
e2e0cb37d7 ci: changed continue-on-error: true to if: succeeded() || failed() 2021-07-18 09:31:23 -07:00
Jarek Kowalski
bdd53f05d5 ci: install provider test dependencies separately 2021-07-18 09:18:14 -07:00
Jarek Kowalski
49d8a32783 testing: split provider tests into separate steps 2021-07-18 09:15:40 -07:00
Jarek Kowalski
33cf322941 ci: run provider test more often to get better stability signal 2021-07-18 09:09:43 -07:00
Jarek Kowalski
42e5661bf4 ci: added license check using github.com/frapposelli/wwhrd (#1200)
* ci: added license check using github.com/frapposelli/wwhrd

* ci: also check NPM licenses

* readme: removed FOSSA badge

* Makefile: install NPM dependencies before license check
2021-07-17 09:58:25 -07:00
Jarek Kowalski
f702e65a31 deps: removed dependency on github.com/minio/minio/pkg, replaced with command line tool (#1185) 2021-07-10 10:15:31 -07:00
Jarek Kowalski
f95ef881ce testing: fixed typo 2021-07-10 08:55:17 -07:00
Jarek Kowalski
a1aa6ae31f tests: install minio mc tool (#1184) 2021-07-10 08:53:54 -07:00
Jarek Kowalski
e64d5b8eab Fixed few subtle threading bugs uncovered by stress test and rewrote the test to be model-based (#1157)
* testing: refactored logs directory management

* content: fixed index mutex to be shared across all write sessions

added mutex protection during writecontent/refresh race

* testing: upload log artifacts

* content: bump revision number after index has been added

This fixes a bug where manifest manager in another session for
the same open repository may not see a content added, because they
will prematurely cache the incomplete set of contents.

This took 2 weeks to find.

* manifest: improved log output, fixed unnecessary mutex release

* testing: rewrote stress test to be model-based and more precise
2021-07-01 21:37:27 -07:00
Jarek Kowalski
e15a79474c azure: added support for using SAS Tokens instead of storage keys (#1093)
Fixes #1071
2021-05-22 12:43:55 -07:00
Jarek Kowalski
5179ad2cd2 cli: test + misc improvements (#1083)
* cli: Added --max-examples-per-bucket flag to 'kopia snapshot estimate'

Added and cleaned up a bunch of unit tests.

Fixes #1054

* cli: misc tests to increase code coverage of the cli package

* ci: move code coverage run into separate GH job
2021-05-17 21:47:11 -07:00
Jarek Kowalski
862298f979 testing: provider test fixes (#1073) 2021-05-15 20:50:28 -07:00
Jarek Kowalski
d81f3f1131 ci: do not upload artifacts from self-hosted runners, fixes build race (#1015)
Fixes #1014
2021-04-25 20:12:25 -07:00
Jarek Kowalski
922bc7cbc8 ci: refactored credentials handling (#987)
This strengthens credential handling after our signing keys may have
been leaked in the [codecov.io breach](https://about.codecov.io/security-update/)

* pass only minimal credentials to each build step to avoid
  exposing sensitive tokens to tools that don't need them
  (like code coverage)
* removed encrypted credential files and replaced with environment-based
* allow full ci/cd including publishing artifacts from forks
* regenerated all passwords, tokens and service accounts
* do not install Google Cloud SDK on GHA - it's already there
* moved RPM signing to 'Stage And Publish Artifacts' phase
* generated new GPG signing key

See https://kopia.discourse.group/t/important-impact-of-codecov-io-security-issue-on-kopia-build-pipeline/377
2021-04-16 08:17:13 -07:00
Jarek Kowalski
52220a75a9 testing: upload endurance test logs as artifacts regardless of status, don't ignore faulures 2021-04-07 14:21:05 -07:00
Jarek Kowalski
7c088338ce testing: upload endurance test logs as artifacts, run more frequently 2021-04-07 14:11:39 -07:00
Jarek Kowalski
3e438dd2b8 ci: run provider-tests as separate GH job (#952) 2021-04-06 21:46:54 -07:00
Jarek Kowalski
79adef0f33 ci: run endurance test 2021-04-03 19:10:36 -07:00
Jarek Kowalski
9a756c719f Enabled race detector in CI, fixed a few data races (#919)
* content: fixed data race in IterateUnreferencedBlobs

* upload: fixed data race between uploader and estimator

* testing: fixed data race in repo/blob/logging test

* makefile: run tests on CI/linux/amd64 with -race

* robustness: fixed test race

* content: fixed data race getContentDataUnlocked that triggers TestParallelWrites - looks scary but in practice very hard to trigger in real life and does not cause data corruption

* testing: reduce test complexity under race detector

* server: fixed minor race in refreshStatus()

* testing: reduced depth of sharedTestDataDir2

* ci: run race detector in separate job

* ci: run unit test race detector in parallel to integration tests
2021-04-02 18:21:04 -07:00
Jarek Kowalski
d0f2ef53d7 blob: improved startup error handling of rclone and webdav PutBlob race (#915)
* added framework for unit testing against remote real rclone remotes,
  added google drive backend
* added parallelism to blobtesting which revealed some races during
  PutBlob with WebDAV.
2021-03-28 08:26:35 -07:00
Jarek Kowalski
b8e89c2808 s3: refactored testing to support multiple S3-compatible providers (#914)
The credentials are passed via JSON matching s3.Options struct
Fixed some verification issue with Wasabi.
2021-03-26 14:33:20 -07:00
Jarek Kowalski
16282b5b61 Makefile refactoring + publish scoop package (#907)
* Makefile: refactoring

- added signing of windows exe in a zip package
- removed Windows build from goreleaser
- removed homebrew from goreleaser
- enabled incremental builds

* ci: publish scoop package
2021-03-22 22:39:24 -07:00
Jarek Kowalski
2e5a3ad3a3 ci: build and publish Homebrew package 2021-03-21 18:41:08 -07:00
Jarek Kowalski
94b3544653 ci: moved Docker installation to 'publish' phase 2021-03-21 16:28:08 -07:00
Jarek Kowalski
3987973964 ci: install credentials before pushing APT/RPM packages 2021-03-21 15:56:23 -07:00