There's now one target, `travis-release` that can be run
locally, or on Travis CI except for:
- code signing (Travis on non-PR runs)
- publishing artifacts to GH releases (Travis on tagged releases)
- creating long-term repository for testing (Travis on tagged releases)
Adding a helper library that wraps fio execution. This is the basic initial check-in that implements the runners, configs, and a single WriteFiles helper. It should be enough to unblock subsequent tasks that will use fio to generate data sets for kopia snapshot verification. More helper workloads can be added as needed.
In this implementation the tests will all skip from test main if the `FIO_EXE` env variable is not set. Adding fio to the CI environment will be addressed as a separate PR.
Tracking progress in issue https://github.com/kopia/kopia/issues/179
Those will make it possible to securely host 'kopia server' embedded
in a desktop app that runs in the background and can access UI.
- added support for using and generating TLS certificates
- added /api/v1/shutdown API to remotely trigger server shutdown
- added support for automatically shutting down server if no requests
arrive in certain amount of time
- added support for generating and printing random password to STDERR
TLS supports 3 modes:
1. serve TLS using externally-provided cert/key PEM files
2. generate & write PEM files, then serve TLS using them
3. generate and use emphemeral cert/key (prints SHA256 fingerprint)
Adding test to probe snapshot failure. Tests snapshot of non-existent source directory, and then iterates through file permissions for files, directories, and issues snapshot to the source. Permission combinations are applied to a parent directory of the source, source itself, contents of source root when the source is a directory, and contents of a subdirectory in that source root.
Fixing kopia executable panic when a directory's contents can't be read. Previously the only Lstat error responded to was not-exist, letting other errors fall through and passing a nil `os.FileInfo` to the following function call, resulting in panic.
```
make lint
kopia/tools/.tools/bin/golangci-lint --deadline 180s run | tee .linterr.txt
cli/command_snapshot_delete.go:5: File is not `goimports`-ed with -local github.com/kopia/kopia (goimports)
tests/end_to_end_test/end_to_end_test.go:303: Function 'TestSnapshotDelete' is too long (187 > 100) (funlen)
func TestSnapshotDelete(t *testing.T) {
tests/end_to_end_test/end_to_end_test.go:306:2: only one cuddle assignment allowed before range statement (wsl)
for _, tc := range []struct {
^
tests/end_to_end_test/end_to_end_test.go:517:4: only one cuddle assignment allowed before if statement (wsl)
if expectDeleteSucceeds {
^
tests/end_to_end_test/end_to_end_test.go:537:2: assignments should only be cuddled with other assignments (wsl)
line := lines[0]
^
tests/end_to_end_test/end_to_end_test.go:542:2: only one cuddle assignment allowed before if statement (wsl)
if typeVal != "policy" {
^
tests/end_to_end_test/end_to_end_test.go:558:2: assignments should only be cuddled with other assignments (wsl)
restoreDir := filepath.Join(e.dataDir, "restored")
^
tests/end_to_end_test/end_to_end_test.go:568:2: if statements should only be cuddled with assignments (wsl)
if got, want := len(si[0].snapshots), 1; got != want {
^
tests/end_to_end_test/end_to_end_test.go:571:2: assignments should only be cuddled with other assignments (wsl)
snapID := si[0].snapshots[0].snapshotID
^
tests/end_to_end_test/end_to_end_test.go:604:2: if statements should only be cuddled with assignments (wsl)
if len(fileInfo) != 0 {
^
cli/command_snapshot_delete.go:25:2: only one cuddle assignment allowed before if statement (wsl)
if err != nil {
^
cli/command_snapshot_delete.go:36:3: if statements should only be cuddled with assignments (wsl)
if labels["username"] != getUserName() {
^
```
Snapshot restore will take a snapshot ID and restore the
associated snapshot to the target path.
- Looks up the manifest with the snapshot ID
- Gets the snapshot root entry
- Copies the snapshot from the root entry to the target path
Because it uses the parent manifest with the copied permissions,
the restored directory will have the permissions of the original
source directory.
Implemented snapshot delete command. Behaves similarly to manifest rm, but with extra verification steps.
- Checks that the referenced manifest is of type "snapshot"
- Checks that the ID points to a snapshot, checks that the host name, user name, and path provided by flag or defaults match the source of the snapshot ID. Command will fail if they do not match, except if given --unsafe-ignore-source, which will bypass the associated safety requirement and delete anyway.
Added end to end tests for input combinations, restore in conjunction with delete, and trying to snapshot delete a manifest by ID of a non-snapshot manifest.
`snapshot gc` marks contents not reachable from the root of any snapshot
as soft-deleted
The algorithm is a mark-and-sweep with parallel iteration of objects.
Currently it stores content IDs and object IDs in a map, so won't scale
to huge repositories, but this can be fixed in the future.
This fixes#110 at least for reasonable repository sizes.
Previously, it was possible for Flush() to miss in-flight writes,
but only when using repository manually since Uploader guarantees
there are no in-flight writes when it completes.
With this change Flush() will guarantee that any pending writes
completed before Flush() has started are guaranteed to be committed
to the repository before Flush() returns.
This was actually a regression introduced in #105.
Added regression test to prevent it from reoccurring.
Repository.Token() generates a base64-encoded token that can
be stored in password manager that fully describes repository connection
information (blob.ConnectionInfo) and optionally a password.
Use `kopia repo status -t` to print the token.
Use `kopia repo status -t -s` to print the token that also includes
repository password.
Use `kopia repo connect from-config --token T` to reconnect using the
token.
Also introduced strongly typed content.ID and manifest.ID (instead of string)
This aligns identifiers across all layers of repository:
blob.ID
content.ID
object.ID
manifest.ID
This updates the terminology everywhere - blocks become blobs and
`storage.Storage` becomes `blob.Storage`.
Also introduced blob.ID which is a specialized string type, that's
different from CABS block ID.
Also renamed CLI subcommands from `kopia storage` to `kopia blob`.
While at it introduced `block.ErrBlockNotFound` and
`object.ErrObjectNotFound` that do not leak from lower layers.
