44 Commits

Author SHA1 Message Date
Jarek Kowalski
044db7593b feat(repository): apply retention policies server-side (#3249)
* feat(repository): apply retention policies server-side

This allows append-only snapshots where the client can never delete
arbitrary manifests and policies are maintained on the server.

The client only needs permissions to create snapshots in a given, which
automatically gives them permission to invoke the server-side method
for their own snapshots only.

* Update cli/command_acl_add.go

Co-authored-by: Guillaume <Gui13@users.noreply.github.com>

* Update internal/server/api_manifest.go

Co-authored-by: Guillaume <Gui13@users.noreply.github.com>

* Update internal/server/api_manifest.go

Co-authored-by: Guillaume <Gui13@users.noreply.github.com>

* Update internal/server/grpc_session.go

Co-authored-by: Guillaume <Gui13@users.noreply.github.com>

---------

Co-authored-by: Guillaume <Gui13@users.noreply.github.com>
2023-09-02 18:23:21 -07:00
Jarek Kowalski
7bda16ab33 feat(repository): introduced fs.UTCTimestamp (#2343)
Fixes #2342
2022-09-02 10:35:59 -07:00
Jarek Kowalski
51dcaa985d chore(ci): upgraded linter to 1.48.0 (#2294)
Mechanically fixed all issues, added `lint-fix` make target.
2022-08-09 06:07:54 +00:00
Jarek Kowalski
30ca3e2e6c Upgraded linter to 1.40.1 (#1072)
* tools: upgraded linter to 1.40.1

* lint: fixed nolintlint vionlations

* lint: disabled tagliatele linter

* lint: fixed remaining warnings
2021-05-15 12:12:34 -07:00
Jarek Kowalski
d2288c443f cli: major refactoring (#1046)
cli: major refactoring of how CLI commands are registered

The goal is to eliminate flags as global variables to allow for better
testing. Each command and subcommand and most sets of flags are now
their own struct with 'setup()' methods that attached the flags or
subcommand to the provided parent.

This change is 94.3% mechanical, but is fully organic and hand-made.

* introduced cli.appServices interface which provides the environment in which commands run
* remove auto-maintenance global flag
* removed globals in memory_tracking.go
* removed globals from cli_progress.go
* removed globals from the update_check.go
* moved configPath into TheApp
* removed remaining globals from config.go
* refactored logfile to get rid of global variables
* removed 'app' global variable
* linter fixes
* fixed password_*.go build
* fixed BSD build
2021-05-03 10:28:00 -07:00
Jarek Kowalski
fa7976599c repo: refactored repository interfaces (#780)
- `repo.Repository` is now read-only and only has methods that can be supported over kopia server
- `repo.RepositoryWriter` has read-write methods that can be supported over kopia server
- `repo.DirectRepository` is read-only and contains all methods of `repo.Repository` plus some low-level methods for data inspection
- `repo.DirectRepositoryWriter` contains write methods for `repo.DirectRepository`

- `repo.Reader` removed and merged with `repo.Repository`
- `repo.Writer` became `repo.RepositoryWriter`
- `*repo.DirectRepository` struct became `repo.DirectRepository`
  interface

Getting `{Direct}RepositoryWriter` requires using `NewWriter()` or `NewDirectWriter()` on a read-only repository and multiple simultaneous writers are supported at the same time, each writing to their own indexes and pack blobs.

`repo.Open` returns `repo.Repository` (which is also `repo.RepositoryWriter`).

* content: removed implicit flush on content manager close
* repo: added tests for WriteSession() and implicit flush behavior
* invalidate manifest manager after write session

* cli: disable maintenance in 'kopia server start'
  Server will close the repository before completing.

* repo: unconditionally close RepositoryWriter in {Direct,}WriteSession
* repo: added panic in case somebody tries to create RepositoryWriter after closing repository
  - used atomic to manage SharedManager.closed

* removed stale example
* linter: fixed spurious failures

Co-authored-by: Julio López <julio+gh@kasten.io>
2021-01-20 11:41:47 -08:00
Jarek Kowalski
5e8e175cfa repo: refactored read/write methods of repo.Repository (#749)
Reader methods go to repo.Reader and write methods go to repo.Writer
Switched usage to new interfaces based on linter errors.
2021-01-04 21:33:12 -08:00
Jarek Kowalski
e03971fc59 Upgraded linter to v1.33.0 (#734)
* linter: upgraded to 1.33, disabled some linters

* lint: fixed 'errorlint' errors

This ensures that all error comparisons use errors.Is() or errors.As().
We will be wrapping more errors going forward so it's important that
error checks are not strict everywhere.

Verified that there are no exceptions for errorlint linter which
guarantees that.

* lint: fixed or suppressed wrapcheck errors

* lint: nolintlint and misc cleanups

Co-authored-by: Julio López <julio+gh@kasten.io>
2020-12-21 22:39:22 -08:00
Jarek Kowalski
9d7cf71a37 Logging flags (#674)
* logging: cleaned up stderr logging

- do not show module
- do not show timestamps by default (enable with --console-timestamps)

* logging: replaced most printStderr() with log.Info

* cli: additional logging cleanup
2020-10-10 10:48:37 -07:00
Jarek Kowalski
29ce1819cb Added support for setting and changing repository client options (description, read-only, hostname, username) (#589)
* repo: refactored client-specific options (hostname,username,description,readonly) into new struct that is JSON-compatible with current config

* cli: added 'repository set-client' to configure parameters of connected repository

* cli: cleaned up 'repository status' output
2020-09-04 13:57:15 -07:00
Jarek Kowalski
6cb9b8fa4f repo: refactored public API (#318)
* This is 99% mechanical:

Extracted repo.Repository interface that only exposes high-level object and manifest management methods, but not blob nor content management.

Renamed old *repo.Repository to *repo.DirectRepository

Reviewed codebase to only depend on repo.Repository as much as possible, but added way for low-level CLI commands to use DirectRepository.

* PR fixes
2020-03-26 08:04:01 -07:00
Jarek Kowalski
e3854f7773 BREAKING: changed how hostname/username are handled
The hostname/username are now persisted when connecting to repository
in a local config file.

This prevents weird behavior changes when hostname is suddenly changed,
such as when moving between networks.

repo.Repository will now expose Hostname/Username properties which
are always guarnateed to be set, and are used throughout.

Removed --hostname/--username overrides when taking snapshot et.al.
2020-02-25 20:40:23 -08:00
Julio Lopez
c03a143ae9 Fix flag description 2020-01-02 09:50:45 -08:00
Jarek Kowalski
6217df1a87 lint: switched to 1.21 and fixed a ton of whitespace issues discovered
by new wsl linter
2019-11-26 06:49:49 -08:00
Pavan Navarathna
829742599c [#124] Add optional flags to override hostname and username 2019-11-22 15:15:21 +09:00
Jarek Kowalski
63303904e1 switched remaining fmt.Errorf to errors.Wrap() 2019-06-01 10:57:05 -07:00
Jarek Kowalski
0c41d41276 Fixed up paths after merge 2019-05-27 15:48:39 -07:00
Jarek Kowalski
327d8317d8 refactored repo/ into separate github.com/kopia/repo/ git repository 2018-10-26 20:40:57 -07:00
Jarek Kowalski
d47f662316 refactoring: moved kopia/kopia/policy to kopia/kopia/snapshot/policy 2018-10-20 09:55:52 -07:00
Jarek Kowalski
3c26be391a added global hidden --timezone flag for changing the timezone in which the timestamps are displayed 2018-10-06 09:54:43 -07:00
Jarek Kowalski
4270d8a67f policy: refactored retention policy to be applied in policy.ApplyRetentionPolicy() 2018-09-16 12:48:07 -07:00
Jarek Kowalski
d8398c4a58 cli: added auto-expiration of snapshots at the end of an upload 2018-09-15 22:17:39 -07:00
Jarek Kowalski
906b8eaee3 plumbed through context to all manifest manager methods and switched to lazy initialization 2018-09-13 15:34:29 -07:00
Jarek Kowalski
6484a55d7c removed policy.Manager and snapshot.Manager and switched to package-level function interface instead 2018-08-30 06:47:58 -07:00
Jarek Kowalski
3ef4593b03 refactored policy management into separate policy/ package 2018-08-29 23:08:40 -07:00
Jarek Kowalski
9a3a2da5fa fixed stdout output of 'expire' 2018-07-27 18:28:58 -07:00
Jarek Kowalski
b2b34c1dea reacted to a change in gometalinter that failed the build 2018-07-10 06:19:22 -07:00
Jarek Kowalski
c57531b751 switched Go logging library to github.com/op/go-logging which lets us override log levels per module 2018-07-09 21:28:12 -07:00
Jarek Kowalski
1defc18300 refactored command_snapshot_expire 2018-06-23 12:30:45 -07:00
Jarek Kowalski
4faf3cd9d0 finalized CLI to manipulate policies 2018-06-16 12:17:58 -07:00
Jarek Kowalski
ec779b14c7 refactored and cleaned up policy management, added retention tags to snapshot list 2018-05-17 20:31:37 -07:00
Jarek Kowalski
d8201229d8 plumbed through ctx in storage.Storage APIs and all uses 2018-04-03 17:39:54 -07:00
Jarek Kowalski
1c4e0e1e47 removed legacy oid format 2018-03-19 19:31:03 -07:00
Jarek Kowalski
453bab3560 fixed some lint errors, mostly dead code and missing error checks 2018-03-19 12:26:28 -07:00
Jarek Kowalski
92330ac247 refactored policy manager to use SourceInfo 2018-02-19 13:20:32 -08:00
Jarek Kowalski
16e44cb37f renamed policy.Manager as snapshot.PolicyManager and policy.Policy as snapshot.Policy 2018-02-19 10:59:15 -08:00
Jarek Kowalski
d97f49285f policy CLI and API cleanup 2018-02-19 10:53:29 -08:00
Jarek Kowalski
10ca234633 switched to using zerolog for logging, by default log level=info, can be set to debug on commandline 2018-01-01 19:12:35 -08:00
Jarek Kowalski
c288daef04 eradicated old metadata.Manager and replaced snapshot.Manager to use manifest.Manager instead 2017-11-27 19:15:02 -08:00
Jarek Kowalski
6e83ad0b9b moved policies to separate package
refactored policy management into policy.Manager that uses manifest.Manager internally
2017-11-27 18:05:32 -08:00
Jarek Kowalski
b5a7d05adf refactored Repository API 2017-09-15 18:35:27 -07:00
Jarek Kowalski
998ed1ddac simplified MetadataManager API for listing, added caching of metadata items (unbounded for now)
disabled go <1.9 since we're now using t.Helper(), which is awesome
2017-08-29 21:46:51 -07:00
Jarek Kowalski
70c9d49232 reorganized more top-level CLI commands into subcommands 2017-08-20 08:00:01 -07:00
Jarek Kowalski
13e328014b moved 'expire' to 'snapshot expire' 2017-08-20 07:42:49 -07:00