42 Commits

Author SHA1 Message Date
Julio Lopez
80eda18f99 chore(ci): enable wsl_v5:err linter (#4928) 2025-11-12 22:15:27 -08:00
Julio López
1f9f9a1846 chore(general): use non-formatting log variants when there is no formatting (#3931)
Use non-formatting logging functions for message without formatting.
For example, `log.Info("message")` instead of `log.Infof("message")`

Configure linter for printf-like functions
2024-06-18 23:13:17 -07:00
Jarek Kowalski
29cd545c33 chore(ci): upgrade linter to 1.56.2 (#3714) 2024-03-09 10:39:11 -08:00
Julio Lopez
c56d330383 feat(cli): handle SIGTERM (#3562)
* refactor(test): allow signaling sub-process from testenv.CLIExeRunner
* test(cli): add test for handling SIGTERM
* feat(general): catch and process SIGTERM for termination
* refactor(cli): rename function cli.App.onTerminate
  Renames function from onCtrlC to a more generic onTerminate
2024-01-11 18:02:31 -08:00
Jarek Kowalski
51dcaa985d chore(ci): upgraded linter to 1.48.0 (#2294)
Mechanically fixed all issues, added `lint-fix` make target.
2022-08-09 06:07:54 +00:00
Jarek Kowalski
8515d050e5 test(infra): improved support for in-process testing (#2169)
* feat(infra): improved support for in-process testing

* support for killing of a running server using simulated Ctrl-C
* support for overriding os.Stdin
* migrated many tests from the exe runner to in-process runner

* added required indirection when defining Envar() so we can later override it in tests

* refactored CLI runners by moving environment overrides to CLITestEnv
2022-07-09 18:22:50 -07:00
Jarek Kowalski
30ca3e2e6c Upgraded linter to 1.40.1 (#1072)
* tools: upgraded linter to 1.40.1

* lint: fixed nolintlint vionlations

* lint: disabled tagliatele linter

* lint: fixed remaining warnings
2021-05-15 12:12:34 -07:00
Jarek Kowalski
d2288c443f cli: major refactoring (#1046)
cli: major refactoring of how CLI commands are registered

The goal is to eliminate flags as global variables to allow for better
testing. Each command and subcommand and most sets of flags are now
their own struct with 'setup()' methods that attached the flags or
subcommand to the provided parent.

This change is 94.3% mechanical, but is fully organic and hand-made.

* introduced cli.appServices interface which provides the environment in which commands run
* remove auto-maintenance global flag
* removed globals in memory_tracking.go
* removed globals from cli_progress.go
* removed globals from the update_check.go
* moved configPath into TheApp
* removed remaining globals from config.go
* refactored logfile to get rid of global variables
* removed 'app' global variable
* linter fixes
* fixed password_*.go build
* fixed BSD build
2021-05-03 10:28:00 -07:00
Jarek Kowalski
f4347886b8 logging: simplified log levels (#954)
Removed Warning, Notify and Fatal:

* `Warning` => `Error` or `Info`
* `Notify` => `Info`
* `Fatal` was never used.

Note that --log-level=warning is still supported for backwards
compatibility, but it is the same as --log-level=error.

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-04-09 07:27:35 -07:00
Robert Kroeger
113670e576 Working kopia mount on Apple Silicon (#825)
* Working kopia mount on Apple Silicon

Work around upstream issues with MacFuse and bazil.org/fuse on Apple
Silicon Macs by using WebDav on darwin arm64. Provides an (at least
temporary) fix for #811.
2021-02-18 09:13:59 -08:00
Jarek Kowalski
fa7976599c repo: refactored repository interfaces (#780)
- `repo.Repository` is now read-only and only has methods that can be supported over kopia server
- `repo.RepositoryWriter` has read-write methods that can be supported over kopia server
- `repo.DirectRepository` is read-only and contains all methods of `repo.Repository` plus some low-level methods for data inspection
- `repo.DirectRepositoryWriter` contains write methods for `repo.DirectRepository`

- `repo.Reader` removed and merged with `repo.Repository`
- `repo.Writer` became `repo.RepositoryWriter`
- `*repo.DirectRepository` struct became `repo.DirectRepository`
  interface

Getting `{Direct}RepositoryWriter` requires using `NewWriter()` or `NewDirectWriter()` on a read-only repository and multiple simultaneous writers are supported at the same time, each writing to their own indexes and pack blobs.

`repo.Open` returns `repo.Repository` (which is also `repo.RepositoryWriter`).

* content: removed implicit flush on content manager close
* repo: added tests for WriteSession() and implicit flush behavior
* invalidate manifest manager after write session

* cli: disable maintenance in 'kopia server start'
  Server will close the repository before completing.

* repo: unconditionally close RepositoryWriter in {Direct,}WriteSession
* repo: added panic in case somebody tries to create RepositoryWriter after closing repository
  - used atomic to manage SharedManager.closed

* removed stale example
* linter: fixed spurious failures

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-01-20 11:41:47 -08:00
Jarek Kowalski
5e8e175cfa repo: refactored read/write methods of repo.Repository (#749)
Reader methods go to repo.Reader and write methods go to repo.Writer
Switched usage to new interfaces based on linter errors.
2021-01-04 21:33:12 -08:00
Jarek Kowalski
e03971fc59 Upgraded linter to v1.33.0 (#734)
* linter: upgraded to 1.33, disabled some linters

* lint: fixed 'errorlint' errors

This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.

Verified that there are no exceptions for errorlint linter which
guarantees that.

* lint: fixed or suppressed wrapcheck errors

* lint: nolintlint and misc cleanups

Co-authored-by: Julio López <julio+gh@kasten.io>
2020-12-21 22:39:22 -08:00
Piotr Tabor
bf729095dc Fuse: Passthrough options to allow other-users access and mounting in empty directory. (#691) 2020-10-31 21:57:27 -07:00
Jarek Kowalski
9d7cf71a37 Logging flags (#674)
* logging: cleaned up stderr logging

- do not show module
- do not show timestamps by default (enable with --console-timestamps)

* logging: replaced most printStderr() with log.Info

* cli: additional logging cleanup
2020-10-10 10:48:37 -07:00
Jarek Kowalski
0758a92c58 restore: improved user experience (#644)
* restore: improved user experience

* 'snapshot restore' is now the same as 'restore' and both will
  support restoring by manifest ID, root ID or root ID + subdirectory

* added support for restoring individual files

* implemented PR feedback and refactored object ID parsing

Moving helpers inside the snapshot/ package helped clean up the code
a lot.
2020-09-28 22:57:24 -07:00
Jarek Kowalski
a5838ff34c Improvements to UX for mounting directories (both CLI and KopiaUI) (#573)
* cli: simplified mount command

See https://youtu.be/1Nt_HIl-NWQ

It will always use WebDAV on Windows and FUSE on Unix. Removed
confusing options.

New usage:

$ kopia mount [--browse]
    Mounts all snapshots in a temporary filesystem directory
    (both Unix and Windows).

$ kopia mount <object> [--browse]
    Mounts given object in a temporary filesystem directory
    (both Unix and Windows).

$ kopia mount <object> z: [--browse]
    Mounts given object as a given drive letter in Windows (using
    temporary WebDAV mount).

$ kopia mount <object> * [--browse]
    Mounts given object as a random drive letter in Windows.

$ kopia mount <object> /mount/path [--browse]
    Mounts given object in given path in Unix.

<object> can be the ID of a directory 'k<hash>' or 'all'

Optional --browse automatically opens OS-native file browser.

* htmlui: added UI for mounting directories

See https://youtu.be/T-9SshVa1d8 for a quick demo.

Also replaced some UI text with icons.

* lint: windows-specific fix
2020-09-03 17:46:48 -07:00
Jarek Kowalski
6cb9b8fa4f repo: refactored public API (#318)
* This is 99% mechanical:

Extracted repo.Repository interface that only exposes high-level object and manifest management methods, but not blob nor content management.

Renamed old *repo.Repository to *repo.DirectRepository

Reviewed codebase to only depend on repo.Repository as much as possible, but added way for low-level CLI commands to use DirectRepository.

* PR fixes
2020-03-26 08:04:01 -07:00
Jarek Kowalski
c8fcae93aa logging: refactored logging
This is mostly mechanical and changes how loggers are instantiated.

Logger is now associated with a context, passed around all methods,
(most methods had ctx, but had to add it in a few missing places).

By default Kopia does not produce any logs, but it can be overridden,
either locally for a nested context, by calling

ctx = logging.WithLogger(ctx, newLoggerFunc)

To override logs globally, call logging.SetDefaultLogger(newLoggerFunc)

This refactoring allowed removing dependency from Kopia repo
and go-logging library (the CLI still uses it, though).

It is now also possible to have all test methods emit logs using
t.Logf() so that they show up in failure reports, which should make
debugging of test failures suck less.
2020-02-25 17:24:44 -08:00
Jarek Kowalski
1a7a02ddbe cleanup imports by grouping all local imports together 2019-06-01 10:57:55 -07:00
Jarek Kowalski
63303904e1 switched remaining fmt.Errorf to errors.Wrap() 2019-06-01 10:57:05 -07:00
Jarek Kowalski
0c41d41276 Fixed up paths after merge 2019-05-27 15:48:39 -07:00
Jarek Kowalski
327d8317d8 refactored repo/ into separate github.com/kopia/repo/ git repository 2018-10-26 20:40:57 -07:00
Jarek Kowalski
c999c0fba8 moved fs/repofs to snapshot/snapshotfs 2018-10-20 11:36:14 -07:00
Jarek Kowalski
6484a55d7c removed policy.Manager and snapshot.Manager and switched to package-level function interface instead 2018-08-30 06:47:58 -07:00
Jarek Kowalski
9060e7d310 moved repository VFS nodes into fs/repofs/ package 2018-08-29 22:56:23 -07:00
Jarek Kowalski
e36d02c59b emit per-dir incomplete reason
store entire rootEntry on snapshot nmanifest
format 'snapshot list' consistently for both root and nested items
2018-05-19 22:00:14 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
c288daef04 eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead 2017-11-27 19:15:02 -08:00
Jarek Kowalski
b5a7d05adf refactored Repository API 2017-09-15 18:35:27 -07:00
Jarek
fa80f40185 added support for mounting snapshots as drive letters and opening them in explorer on Windows. 2017-09-05 19:01:51 -07:00
Jarek Kowalski
76b712c2fb moved creation of repofs entries to snapshot.Manager 2017-09-04 20:47:51 -07:00
Jarek Kowalski
371fc15694 merged 'repofs' into 'snapshot' package 2017-09-04 20:26:25 -07:00
Jarek Kowalski
2b032b1748 refactored FS caching flags 2017-09-04 17:49:36 -07:00
Jarek Kowalski
b08a35e92b refactored FS cache into reusable wrapper that can be applied on top of any filesystem abstraction 2017-09-04 17:42:50 -07:00
Jarek Kowalski
4b1547fec9 periodically refresh the metadata manager cache when mounting 2017-08-29 21:46:51 -07:00
Jarek Kowalski
d613d99f7e beginnings of WebDAV mounting support, not really working yet, need to do have better caching 2017-08-27 15:02:56 -07:00
Jarek Kowalski
ff487d1e85 added ability to mount special object 'all' that exposes the whole set of snapshots grouped by user@host, path and time 2017-08-19 19:49:18 -07:00
Jarek Kowalski
cfbef13aff Major refactoring to how repository connections are established and managed to simplify programmatic API usage and CLI.
Removed unnecessary and confusing options, improved the API.
2017-08-01 08:35:14 +02:00
Jarek Kowalski
095fc4aba4 removed 'client' package and merged with 'repo', names still not final 2017-08-01 08:29:42 +02:00
Jarek Kowalski
4dcdb69dcc replaced use of Vault with Repository 2017-08-01 08:29:41 +02:00
Jarek Kowalski
4a878738a0 moved main executable package to kopia/kopia, moved cli support to kopia/kopia/cli 2017-07-17 15:20:32 +02:00