Commit Graph

1051 Commits

Author SHA1 Message Date
Niraj Tolia
0559cb8a71 Spelling fixes 2020-02-05 21:38:16 -08:00
Niraj Tolia
e51e4e44fe Make Kopia-specific changes (inspired by Vitess) 2020-02-05 21:38:16 -08:00
Niraj Tolia
0ba102cfa7 Add Governance model
Import from http://oss-watch.ac.uk/resources/meritocraticgovernancemodel
2020-02-05 21:38:16 -08:00
Niraj Tolia
b6095c0e61 Add Code of Conduct 2020-02-05 21:38:16 -08:00
Jarek Kowalski
758dbccd01 kopia-ui: UI/API server security (TLS & base auth)
This automatically launches Kopia CLI with new flags:

`--tls-generate-cert` which generates in-memory TLS certs for the server
and prints its fingerprint to stderr.

`--random-password` which sets up random base auth password

KopiaUI will then listen to the stderr messages and parse the cert
fingerprint and password and automatically provide them in the browser
it opens to show the UI.

This ensures that any local process (or script in a browser, etc.)
other than KopiaUI itself won't have access to the server API nor
UI because they won't know the ephemeral random password and won't
be able to intercept it due to TLS protection.
2020-02-05 17:08:47 -08:00
Jarek Kowalski
0a42ceab27 gcs: added --embed-credentials options which persists --credentials-file as part of Kopia configuration 2020-02-04 22:35:26 -08:00
Jarek Kowalski
0e9e5fe7a7 travis: only deploy website from linux job 2020-02-03 22:48:05 -08:00
Jarek Kowalski
d2360327f8 travis: caching 2020-02-03 19:08:38 -08:00
Jarek Kowalski
80f5a403ce travis: only build the UI on macOS since Linux will build everything except it 2020-02-03 19:08:38 -08:00
Jarek Kowalski
6493a56270 travis: fixed --skip-sign logic 2020-02-02 22:23:08 -08:00
Jarek Kowalski
2b07045ef9 travis: refactored release to be easier to understand
There's now one target, `travis-release` that can be run
locally, or on Travis CI except for:

- code signing (Travis on non-PR runs)
- publishing artifacts to GH releases (Travis on tagged releases)
- creating long-term repository for testing (Travis on tagged releases)
2020-02-02 22:23:08 -08:00
Jarek Kowalski
390f7904e3 changelog: disable sorting 2020-02-01 17:11:09 -08:00
Jarek Kowalski
03d7abc619 travis: add caching of electron artifacts 2020-02-01 16:59:13 -08:00
Jarek Kowalski
a454f04d75 travis: fixed docker variable filtering which was breaking kopia-ui 2020-02-01 16:59:13 -08:00
Jarek Kowalski
1e054554a3 kopia-ui: pass missing environment variables to Electron build on Travis 2020-02-01 13:20:18 -08:00
Jarek Kowalski
bc4f1f76fe kopia-ui: pass missing environment variables to Electron build on Travis 2020-02-01 12:55:51 -08:00
Jarek Kowalski
fc5c68f953 readme: slack badge 2020-02-01 12:37:25 -08:00
Jarek Kowalski
c3ead4bc3e Kopia UI: added desktop app shell based on Electron that runs in the tray (#183)
* app: added desktop app shell based on Electron that runs in the tray, starts a background kopia server and allows access to the UI

* icons: updated icons for the app

* htmlui: flexible containers
2020-02-01 11:58:22 -08:00
Jarek Kowalski
e9ad7b0e14 site: updated slack invite (#185)
* slack: switched to slack.kopia.io link
2020-02-01 11:39:05 -08:00
Nick
edbc7591d7 [robustness testing] Adding library for file generation using 'fio'
Adding a helper library that wraps fio execution. This is the basic initial check-in that implements the runners, configs, and a single WriteFiles helper. It should be enough to unblock subsequent tasks that will use fio to generate data sets for kopia snapshot verification. More helper workloads can be added as needed.

In this implementation the tests will all skip from test main if the `FIO_EXE` env variable is not set. Adding fio to the CI environment will be addressed as a separate PR.

Tracking progress in issue https://github.com/kopia/kopia/issues/179
2020-01-30 21:37:21 -08:00
Julio Lopez
782ac3a228 Change import path to minio-go/v6
Follows the recommendations for importing dependencies with versions >= 2

https://tip.golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning
2020-01-30 21:18:04 -08:00
Nick
08519b4ab0 Add test to induce a failure during a restore using blob delete
Add a test that will delete a kopia pack blob, expecting a subsequent restore issued from the snapshot that referenced that blob to fail.
2020-01-28 19:36:00 -08:00
Jarek Kowalski
9680dc376b cli: improvements for 'kopia server' and client
Those will make it possible to securely host 'kopia server' embedded
in a desktop app that runs in the background and can access UI.

- added support for using and generating TLS certificates
- added /api/v1/shutdown API to remotely trigger server shutdown
- added support for automatically shutting down server if no requests
  arrive in certain amount of time
- added support for generating and printing random password to STDERR

TLS supports 3 modes:

1. serve TLS using externally-provided cert/key PEM files
2. generate & write PEM files, then serve TLS using them
3. generate and use emphemeral cert/key (prints SHA256 fingerprint)
2020-01-24 17:25:45 -08:00
Nick
7367f85afe Snapshot failure test and fix kopia panic for non-executable directories
Adding test to probe snapshot failure. Tests snapshot of non-existent source directory, and then iterates through file permissions for files, directories, and issues snapshot to the source. Permission combinations are applied to a parent directory of the source, source itself, contents of source root when the source is a directory, and contents of a subdirectory in that source root.

Fixing kopia executable panic when a directory's contents can't be read. Previously the only Lstat error responded to was not-exist, letting other errors fall through and passing a nil `os.FileInfo` to the following function call, resulting in panic.
2020-01-20 21:29:55 -08:00
Nick
3913241159 [Trivial] Fix command description for blob delete
Fix probable copy paste error on "blob delete" subcommand from
the "blob show" description.
2020-01-20 21:28:44 -08:00
Jarek Kowalski
2b475565bb testing: marked all remaining tests as parallel 2020-01-16 19:40:34 -08:00
Jarek Kowalski
2bc8383e47 testing: customized e2e test directory tree shapes
based on PR feedback, instead of 3 uniform directories,
have 3 different shapes.
2020-01-16 19:40:34 -08:00
Jarek Kowalski
7d39be976f testing: split end_to_end_test into separate files.
refactored test helpers to separate package
made all tests parallel and improved the code structure
2020-01-16 19:40:34 -08:00
Jarek Kowalski
644ef93d45 s3: added optional --region parameter
Fixes #168
v0.4.0
2020-01-09 16:37:36 -08:00
Jarek Kowalski
0b8c4d0ef9 object: fixed compression bug where we were not clearing the buffer
this effectively defeated the purpose of compression, caused high
memory usage and other kinds of bad behavior.

refactored the code to prevent this issue by resetting the buffer
at the caller not callee.

fixed previous e2e test to catch the issue mentioned in #166,
verified it fails against master and passes with this change.
2020-01-09 16:36:57 -08:00
Jarek Kowalski
de71f0f057 fs: added functions Update() and Remove() on fs.Entries which return a copy of entries with specified entry updated or removed 2020-01-05 09:13:41 -08:00
Jarek Kowalski
aa1b24762d lint: added retry loop around linter invocations 2020-01-03 16:39:30 -08:00
Jarek Kowalski
1bb7eeebe7 fixed inadvertent behavior change 2020-01-03 16:39:30 -08:00
Jarek Kowalski
ac70a38101 lint: upgraded to 1.22.2 and make lint issues a build failure
fixed or silenced linter warnings, mostly due to magic numeric constants
2020-01-03 16:39:30 -08:00
Julio Lopez
b72965444c Update e2e restore test 2020-01-02 13:49:28 -08:00
Julio Lopez
1249a255d3 Allow copying to an existing directory
Add some logging
2020-01-02 13:49:28 -08:00
Julio Lopez
ac6e84107e Add CopyOptions and copier structs 2020-01-02 13:49:28 -08:00
Julio Lopez
20b896da18 Add restore command flags 2020-01-02 13:49:28 -08:00
Julio Lopez
7361d277e0 Remove special-case handling when target is root directory
No longer needed since directories can be overwritten by default
2020-01-02 13:49:28 -08:00
Julio Lopez
99bca0218d Add comment to exported constant 2020-01-02 09:50:45 -08:00
Julio Lopez
8526376e08 Close channel in the error case as well 2020-01-02 09:50:45 -08:00
Julio Lopez
2389abb465 manifest.TypeLabelKey const 2020-01-02 09:50:45 -08:00
Julio Lopez
5c62bce5eb snapshot.ManifestType const 2020-01-02 09:50:45 -08:00
Julio Lopez
c0b1345d9a Trivial: add clarifying comment 2020-01-02 09:50:45 -08:00
Julio Lopez
8e9d56fcf2 Trivial: simplify localfs.Directory 2020-01-02 09:50:45 -08:00
Julio Lopez
a7ee7d57b6 Install go-bindata in /bin 2020-01-02 09:50:45 -08:00
Julio Lopez
c03a143ae9 Fix flag description 2020-01-02 09:50:45 -08:00
Jarek Kowalski
5d0167e2bb snapshotfs: added HasDirEntry and type assertions 2020-01-01 15:10:43 -08:00
Jarek Kowalski
8ff8ec0a36 snapshotfs: expose functions to round-trip between fs.Entry and DirEntry 2020-01-01 15:10:43 -08:00
Jarek Kowalski
503fa74be5 fs: added Directory.Child() method to look up children by name without necessarily loading all of them 2019-12-30 19:21:17 -08:00