Commit Graph

11 Commits

Author SHA1 Message Date
Z
33c8733750 add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371) 2021-10-09 12:38:36 -07:00
Jarek Kowalski
35d0f31c0d huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)
This helps recycle buffers more efficiently during snapshots.
Also, improved memory tracking, enabled profiling flags and added pprof
by default.
2021-08-20 08:45:10 -07:00
Jarek Kowalski
4c9b9e523a Test cleanup improvements (#1203)
* testing: removed testutil.Retry because all providers now have internal retries

* testing: simplified and unified cleanup for all cloud providers using shared buckets
2021-07-17 18:54:37 -07:00
Jarek Kowalski
f702e65a31 deps: removed dependency on github.com/minio/minio/pkg, replaced with command line tool (#1185) 2021-07-10 10:15:31 -07:00
Jarek Kowalski
62ad437bb6 Implemented epoch-based index manager (#1174)
* epoch: misc fixes and logging

* blob: misc helpers

* cli: removed useless 'repository upgrade', replaced by 'repository set-parameters'

* content: implemented indexBlobManagerV1 which uses epoch manager

* cli: commands to manipulate repository epoch parameters

* cli: commands to examine epoch-based indexes

* content: added test suite that uses epoch-based index manager

* content: fixed a ton of test data races caused by sharing blobtesting.DataMap

* cli: additional tests and validation for 'repository set-params'

* testing: replaced the use of suite with our own, since suite is not parallelizable
2021-07-06 21:38:08 -07:00
Jarek Kowalski
281a7fcc95 e2e test refactoring (#1058)
* tests: refactored test directory creation into separate package

* mechanical: refactored e2e test output parsing and error handling
2021-05-08 11:15:31 -07:00
Jarek Kowalski
3e438dd2b8 ci: run provider-tests as separate GH job (#952) 2021-04-06 21:46:54 -07:00
Jarek Kowalski
9a756c719f Enabled race detector in CI, fixed a few data races (#919)
* content: fixed data race in IterateUnreferencedBlobs

* upload: fixed data race between uploader and estimator

* testing: fixed data race in repo/blob/logging test

* makefile: run tests on CI/linux/amd64 with -race

* robustness: fixed test race

* content: fixed data race getContentDataUnlocked that triggers TestParallelWrites - looks scary but in practice very hard to trigger in real life and does not cause data corruption

* testing: reduce test complexity under race detector

* server: fixed minor race in refreshStatus()

* testing: reduced depth of sharedTestDataDir2

* ci: run race detector in separate job

* ci: run unit test race detector in parallel to integration tests
2021-04-02 18:21:04 -07:00
Jarek Kowalski
7c108930ef testing: ensure tests are releasing all buffer pools to reduce memory usage, we had huge leaks (#895)
* testing: ensure tests are releasing all buffer pools to reduce memory usage, we had huge leaks

* object: reduced complexity and memory usage of TestEndToEndReadAndSeekWithCompression

* manifest: more test fixes

* trivial: update comment

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-03-18 06:40:33 -07:00
Jarek Kowalski
e2b9a81ac3 Major CI/CD refactoring and re-added support for ARM/ARM64 runners (#849)
* ci: refactored CI/CD logic & Makefile

- removed all travis CI emulation environment variables and replaced with:

CI_TAG=<empty>|tag
IS_PULL_REQUEST=false|true

- refactored all OS and architecture-specific decisions to use around standard GOOS/GOARCH values instead of uname/OS
- re-added self-hosted runner for ARMHF (3 replicas)
- added brand new self-hosted runner for ARM64 (3 replicas)
- disabled attempts to publish and sign on forks
- improved integration test log output to better see timings and sub-tests
- print longest tests (unit tests and integration) after each run
- verified that all configurations build successfully on a clone (jkowalski/kopia)
- run make setup in parallel

* testing: fixed tests on ARM and ARM64

- fixed ARM-specific alignment issue
- cleaned up test logging
- fixed huge params warning threshold because it was tripping on ARM.
- reduced test complexity to make them fit in 15 minutes
2021-02-23 00:52:54 -08:00
Jarek Kowalski
77691cf0a2 sftp: run private SFTP server based on Docker (atmoz/sftp) (#796)
* sftp: run private SFTP server based on Docker (atmoz/sftp)

* sftp: disable test on GH Actions && !linux because it does not support Docker.
2021-01-25 22:13:30 -08:00