Commit Graph

49 Commits

Author SHA1 Message Date
Julio Lopez
8706239a9f build(infra): checkout repo before setting up Go (#2903)
This way GH caching for Go packages has a chance to work.
See https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs

* build(providers): checkout repo before setting up Go
* build(infra): checkout repo before setting up Go: apply to other workflows as well.
2023-04-07 17:22:05 -04:00
Julio Lopez
d5d150104f chore(build): upgrade to Go 1.20 (#2876) 2023-03-31 07:58:37 -07:00
dependabot[bot]
5a25affd2a build(deps): bump actions/setup-go from 3 to 4 (#2841)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 04:45:19 +00:00
Jarek Kowalski
f69424961f chore(ci): upgrade golang to 1.19.2 and linter to 1.50.1 (#2526)
Lack of generics support is blocking various dependency upgrades,
so this unblocks that.

Temporarily disabled `checklocks` linter until it is fixed upstream.
2022-10-28 11:02:47 -07:00
Jarek Kowalski
f1ffc0625c chore(ci): temporarily disable ARMHF runners (#2083)
The docker images we used were discontinued upstream, we either need to:

* drop 32-bit ARM validation, keep publishing builds since they
  are cross-compiled from AMD64 Linux anyway

* drop 32-bit ARM validation and releases altogether, looking at stats
  there are only tiny amounts of 7-day active users who pull from the
  APT repository

* find a replacement for this runner
2022-06-24 21:30:58 -07:00
Jarek Kowalski
ebd8f113c6 chore(ci): set golang version explicitly to 1.18 (#1952)
We were unintentionally doing that because ^1.17 was pulling 1.18.x
2022-05-16 11:55:04 -07:00
Julio Lopez
8567db6c88 build(deps): bump docker actions from 1 to 2 (#1947)
* build(deps): bump docker/setup-qemu-action from 1 to 2

Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump docker/setup-buildx-action from 1 to 2

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 00:03:35 +00:00
Julio Lopez
e1c74a5dab build(deps): bump actions/download-artifact from 2 to 3 (#1893)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-19 05:07:37 +00:00
Julio Lopez
6fa621941f build(deps): bump actions/upload-artifact from 2 to 3 (#1889)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-13 23:05:08 +00:00
Julio Lopez
83bc416aeb build(deps): bump actions/setup-go from 2 to 3 (#1888)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2 to 3.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-13 22:25:48 +00:00
Jarek Kowalski
766cb57160 feat(ci): automatically bump homebrew version on tag (#1807) 2022-03-06 21:44:13 -08:00
dependabot[bot]
c7ccf4c79d build(deps): bump actions/checkout from 2 to 3 (#1782)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 21:22:55 -08:00
Jarek Kowalski
aa6f9b17df build(ci): auto-cancel previous workflows on the same branch (#1747) 2022-02-11 18:21:11 -08:00
Jarek Kowalski
09c1489be4 fix(ci): add rclone binaries to the list of artifacts (#1719) 2022-02-02 21:53:39 -08:00
Jarek Kowalski
c129f28b16 chore(ci): moved tests to standalone GHA workflow that runs in parallel (#1663) 2022-01-15 12:12:18 -08:00
Jarek Kowalski
daacc6a8f4 htmlui: moved HTML UI to separate repository (#1600)
The source code for htmlui is now in github.com/kopia/htmlui
GitHub Actions will drop compiled builds in github.com/kopia/htmluibuild
where they are now used as a go module dependency.

This greatly simplifies the build and improves the security,
because Kopia will be consuming pre-built htmlui.

This also means kopia can now installed with embedded UI using:
`go install github.com/kopia/kopia@latest`
2021-12-18 12:28:30 -08:00
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Julio Lopez
6472d171d4 Skip publishing artifacts on fork repos (#1364) 2021-10-07 08:42:40 -07:00
Jarek Kowalski
cc692c40af ci: delete golang installation on armhf before installing a new one (#1276)
Per https://github.com/golang/go/issues/48134#issuecomment-911431265
2021-09-02 19:52:49 -07:00
Jarek Kowalski
06451aa7c2 upgrade golang to 1.17 + various dependencies (#1256) 2021-09-01 17:08:09 -07:00
Jarek Kowalski
e64d5b8eab Fixed few subtle threading bugs uncovered by stress test and rewrote the test to be model-based (#1157)
* testing: refactored logs directory management

* content: fixed index mutex to be shared across all write sessions

added mutex protection during writecontent/refresh race

* testing: upload log artifacts

* content: bump revision number after index has been added

This fixes a bug where manifest manager in another session for
the same open repository may not see a content added, because they
will prematurely cache the incomplete set of contents.

This took 2 weeks to find.

* manifest: improved log output, fixed unnecessary mutex release

* testing: rewrote stress test to be model-based and more precise
2021-07-01 21:37:27 -07:00
Jarek Kowalski
5179ad2cd2 cli: test + misc improvements (#1083)
* cli: Added --max-examples-per-bucket flag to 'kopia snapshot estimate'

Added and cleaned up a bunch of unit tests.

Fixes #1054

* cli: misc tests to increase code coverage of the cli package

* ci: move code coverage run into separate GH job
2021-05-17 21:47:11 -07:00
Jarek Kowalski
d81f3f1131 ci: do not upload artifacts from self-hosted runners, fixes build race (#1015)
Fixes #1014
2021-04-25 20:12:25 -07:00
Jarek Kowalski
922bc7cbc8 ci: refactored credentials handling (#987)
This strengthens credential handling after our signing keys may have
been leaked in the [codecov.io breach](https://about.codecov.io/security-update/)

* pass only minimal credentials to each build step to avoid
  exposing sensitive tokens to tools that don't need them
  (like code coverage)
* removed encrypted credential files and replaced with environment-based
* allow full ci/cd including publishing artifacts from forks
* regenerated all passwords, tokens and service accounts
* do not install Google Cloud SDK on GHA - it's already there
* moved RPM signing to 'Stage And Publish Artifacts' phase
* generated new GPG signing key

See https://kopia.discourse.group/t/important-impact-of-codecov-io-security-issue-on-kopia-build-pipeline/377
2021-04-16 08:17:13 -07:00
Jarek Kowalski
d0f2ef53d7 blob: improved startup error handling of rclone and webdav PutBlob race (#915)
* added framework for unit testing against remote real rclone remotes,
  added google drive backend
* added parallelism to blobtesting which revealed some races during
  PutBlob with WebDAV.
2021-03-28 08:26:35 -07:00
Jarek Kowalski
b8e89c2808 s3: refactored testing to support multiple S3-compatible providers (#914)
The credentials are passed via JSON matching s3.Options struct
Fixed some verification issue with Wasabi.
2021-03-26 14:33:20 -07:00
Jarek Kowalski
16282b5b61 Makefile refactoring + publish scoop package (#907)
* Makefile: refactoring

- added signing of windows exe in a zip package
- removed Windows build from goreleaser
- removed homebrew from goreleaser
- enabled incremental builds

* ci: publish scoop package
2021-03-22 22:39:24 -07:00
Jarek Kowalski
2e5a3ad3a3 ci: build and publish Homebrew package 2021-03-21 18:41:08 -07:00
Jarek Kowalski
94b3544653 ci: moved Docker installation to 'publish' phase 2021-03-21 16:28:08 -07:00
Jarek Kowalski
3987973964 ci: install credentials before pushing APT/RPM packages 2021-03-21 15:56:23 -07:00
Jarek Kowalski
335ff79e78 CI refactoring (#903)
* ci: publish additional binaries as artifacts

* ci: install {htmlui,app}/node_modules as part of ci-setup

* ci: added retry to improve robustness of builds

* ci: continue-on-error for non-pull-requests

* ci: moved publish-packages to Stage And Publish Artifacts phase
2021-03-21 15:17:39 -07:00
Jarek Kowalski
97eb7b23dc ci: store changelog along with artifacts 2021-03-20 15:10:53 -07:00
Jarek Kowalski
10774d5acd ci/cd: refactored how GH releases are published (#867)
- stop publishing to github directly from goreleaser and electron builder
- upload dist/ as artifact at the end of each build
- stage release in a separate directory, generated checksum, GPG sign and publish
- removed spaces from Windows setup exe
2021-03-20 11:46:05 -07:00
Jarek Kowalski
3b6cf5cc7b docker: publish docker images to docker hub (#896)
docker: push dockerhub tags

* major.minor - tracks latest major.minor release (except major==0)
* latest - tracks official stable releases
* testing - tracks official stable releases or pre-releases (beta, rc)
* unstable - tracks nightly/unstable builds
2021-03-19 12:51:24 -07:00
Jarek Kowalski
dd740d42e6 Revert "ci: run build every 4 hours to get more test samples (temporary)"
This reverts commit e4bb295a1c.
2021-03-16 22:32:44 -07:00
Jarek Kowalski
e4bb295a1c ci: run build every 4 hours to get more test samples (temporary) 2021-03-16 08:45:58 -07:00
Jarek Kowalski
5928df3498 Makefile: build fat macOS binary (amd64 and arm64) (#866)
This will be embedded in KopiaUI which is built on macOS runner.
2021-03-04 19:35:26 -08:00
Jarek Kowalski
e2b9a81ac3 Major CI/CD refactoring and re-added support for ARM/ARM64 runners (#849)
* ci: refactored CI/CD logic & Makefile

- removed all travis CI emulation environment variables and replaced with:

CI_TAG=<empty>|tag
IS_PULL_REQUEST=false|true

- refactored all OS and architecture-specific decisions to use around standard GOOS/GOARCH values instead of uname/OS
- re-added self-hosted runner for ARMHF (3 replicas)
- added brand new self-hosted runner for ARM64 (3 replicas)
- disabled attempts to publish and sign on forks
- improved integration test log output to better see timings and sub-tests
- print longest tests (unit tests and integration) after each run
- verified that all configurations build successfully on a clone (jkowalski/kopia)
- run make setup in parallel

* testing: fixed tests on ARM and ARM64

- fixed ARM-specific alignment issue
- cleaned up test logging
- fixed huge params warning threshold because it was tripping on ARM.
- reduced test complexity to make them fit in 15 minutes
2021-02-23 00:52:54 -08:00
Julio López
48fe5e96b8 Upgrade to Go 1.16 (#842)
* Use 1.16 in GHA workflows
* Require Go 1.16 in `go.mod`
2021-02-17 21:52:23 -08:00
Jarek Kowalski
fef23a7e2c disable code coverage 2021-02-14 21:45:56 -08:00
Jarek Kowalski
0b1d6d3159 codecov: run only on ubuntu 2021-02-14 21:18:09 -08:00
Jarek Kowalski
0581a07220 coverage: use GH action to upload coverage 2021-02-14 20:45:26 -08:00
Jarek Kowalski
70b0d8d9c9 integrate with codecov.io (#836)
* integrate with codecov.io
2021-02-14 20:17:05 -08:00
Jarek Kowalski
a655d7b9e2 minor build tweaks (#830) 2021-02-12 18:36:01 -08:00
Jarek Kowalski
38fc17bae8 ci: changed cron schedule to run once a week, instead of twice daily 2021-01-24 09:02:15 -08:00
Jarek Kowalski
3cce502017 ci: disabled armhf runner until it's fixed 2021-01-14 20:51:12 -08:00
Jarek Kowalski
fd0e8ed2c9 ci: fix coveralls publishing 2020-12-24 11:24:31 -08:00
Jarek Kowalski
2541f0013c ci: GitHub actions support (#739)
This mimics Travis behavior and also allows making full local release
from a GH owner other than 'kopia' as long as required secrets are
provided.

Supports:

- basic testing on PRs
- publishing kopia RPM, DEB and TAR.GZ release to <owner>/kopia
  (on tags)
- publishing KopiaUI release to <owner>/kopia (on tags)
  or <owner>/kopia-ui-release (on master)
- publishing RPM and APT packages from official kopia/kopia and
  master branch only

Verified on jkowalski/kopia clone:

Tags:
https://github.com/jkowalski/kopia/actions/runs/439735808
https://github.com/jkowalski/kopia/releases/tag/v0.8.0-testing4

Post-submit:
https://github.com/jkowalski/kopia/actions/runs/439720039
https://github.com/jkowalski/kopia-ui-release/releases/tag/v20201222.0.225118

* ci: remove .travis.yml
* testing: skip tests if environment values are empty
* ci: run heavyweight tests on amd64 only
* ci: remove .appveyor.yml
* makefile: removed appveyor travis emulation
2020-12-24 01:11:57 -08:00
Jarek Kowalski
213a557435 ci: run make on GitHub actions (#736) 2020-12-21 17:15:46 -08:00