Jarek Kowalski 2e9a57f0b4 server: support for server control APIs and tooling (#1644) ...

This adds new set of APIs `/api/v1/control/*` which can be used to administratively control a running server.

Once the server is started, the administrative user can control it
using CLI commands:

export KOPIA_SERVER_ADDRESS=...
export KOPIA_SERVER_CERT_FINGERPRINT=...
export KOPIA_SERVER_PASSWORD=...

* `kopia server status` - displays status of sources managed by the server
* `kopia server snapshot` - triggers server-side upload of snapshots for managed sources
* `kopia server cancel` - cancels upload of snapshots for managed sources
* `kopia server pause` - pauses scheduled snapshots for managed sources
* `kopia server resume` - resumes scheduled snapshots for managed sources
* `kopia server refresh` - causes server to resynchronize with externally-made changes, such as policies or new sources
* `kopia server flush` - causes server to flush all pending writes
* `kopia server shutdown` - graceful shutdown of the server

Authentication uses new user `server-control` and is disabled
by default. To enable it when starting the server, provide the password
using one of the following methods:

* `--server-control-password`
* `--random-server-control-password`
* `.htpasswd` file
* `KOPIA_SERVER_CONTROL_PASSWORD` environment variable

This change allows us to tighten the API security and remove some
methods that UI user was able to call, but which were not needed.

2022-01-03 18:48:38 -08:00

..

app.go

logging: added log rotation and improved predictability of log sweep (#1562)

2021-12-03 16:43:46 -08:00

auto_upgrade.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

cli_progress.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_acl_add.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_acl_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_enable.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_acl.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_benchmark_compression.go

cli: benchmark compressor memory usage (#1416)

2021-10-18 21:39:00 -07:00

command_benchmark_crypto.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

command_benchmark_splitters.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_benchmark_test.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_benchmark.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_blob_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_gc.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_list.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

command_blob_shards_modify_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_blob_shards_modify.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_blob_shards.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_blob_show_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_blob_show.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

command_blob_stats.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_blob.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_cache_clear.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_cache_info.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_cache_set_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_cache_set.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_cache_sync_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_cache_sync.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_cache.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_content_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_content_list.go

Support for content-level compression (#1076)

2021-05-22 05:35:27 -07:00

command_content_range_flags.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_content_rewrite.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_content_show.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_content_stats.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_content_verify_test.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_content_verify.go

Fix endurance test (#1254)

2021-08-27 04:22:18 -07:00

command_content.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_diff.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_index_epoch_list.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_epoch.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_inspect_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_index_inspect.go

cli: added 'index inspect --parallel' flag (#1404)

2021-10-17 18:35:21 -07:00

command_index_list.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_optimize.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_index_recover.go

cli: optimized 'kopia index recover' by leveraging partial read and parallelism (#1094)

2021-05-23 22:18:31 -07:00

command_index.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_logs_cleanup.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_list.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_session.go

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

command_logs_show.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_logs_test.go

maintenance: send logs to content log as well (#1496)

2021-11-06 23:08:00 -07:00

command_logs.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_ls.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_maintenance_info_test.go

cli: include parameters in maintenance info JSON output (#981)

2021-09-10 17:51:55 -07:00

command_maintenance_info.go

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

command_maintenance_run.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

command_maintenance_set.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_maintenance.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_manifest_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_manifest_show.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_mount.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_policy_edit.go

Fix typo in policy help documentation (#1597)

2021-12-18 11:17:27 -08:00

command_policy_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_policy_remove.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_actions.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

command_policy_set_compression.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_error_handling.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_files.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_logging_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_logging.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_retention.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_scheduling.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_show.go

policy: revamped policy merge mechanism (#1538)

2021-11-27 18:14:45 -08:00

command_policy.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_repository_change_password_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_repository_change_password.go

Repository password change support (#1197)

2021-07-17 07:58:02 -07:00

command_repository_connect_from_config.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_repository_connect_server.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_connect.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_repository_create.go

policy: revamped policy merge mechanism (#1538)

2021-11-27 18:14:45 -08:00

command_repository_disconnect.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_repair.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_repository_set_client.go

repo: limit the duration of kopia.repository caching to 15 minutes (#1196)

2021-07-15 11:32:55 -07:00

command_repository_set_parameters_test.go

content: fixed repo upgrade version (#1286)

2021-09-10 22:51:51 -07:00

command_repository_set_parameters.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_repository_status.go

fixed directory read race condition (#1489)

2021-11-05 10:18:03 -07:00

command_repository_sync.go

blob: replaced blob.Storage.SetTime() method with blob.PutOptions.SetTime (#1595)

2021-12-18 14:00:20 -08:00

command_repository_validate_provider.go

Added 'kopia repository validate-provider` (#1205)

2021-07-19 21:42:24 -07:00

command_repository.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

command_restore.go

[snapshot restore] use non-atomic writes (#1534)

2021-11-26 13:10:44 -08:00

command_server_cancel.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_control_test.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_flush.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_pause.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_refresh.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_resume.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_shutdown.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_snapshot.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_source_manager_action.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_start.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_status.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_tls.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_session_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_session.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_show.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_snapshot_copy_move_history.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_create.go

cli: reduce number of index blob writes in 'snapshot create --all' (#1574)

2021-12-11 10:08:34 -08:00

command_snapshot_delete.go

cli: added 'snapshot delete --all-snapshots-for-source' (#1397)

2021-10-17 09:46:44 -07:00

command_snapshot_estimate_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_snapshot_estimate.go

cli: test + misc improvements (#1083)

2021-05-17 21:47:11 -07:00

command_snapshot_expire.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_snapshot_gc.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_list.go

Merge retention tags in snapshot lists (#1567)

2021-12-05 20:49:41 -08:00

command_snapshot_migrate.go

cli: when migrating snapshot honor destination policies (#1433)

2021-10-21 22:12:06 -07:00

command_snapshot_pin_test.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_snapshot_pin.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_snapshot_restore.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_verify.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_snapshot.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_user_add_set.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_user_info.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

config.go

introduced structural debug logging and optional JSON output (#1475)

2021-11-03 21:57:37 -07:00

inproc.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

json_output.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

memory_tracking.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

password_darwin.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

password_linux.go

cli: linux: read KOPIA_USE_KEYRING env (#1337)

2021-10-01 22:56:50 -07:00

password_other.go

linter: upgrade to 1.42.1 (#1292)

2021-09-14 19:11:39 -07:00

password_windows.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

password.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

profile.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

show_utils.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

sighup_unix.go

linter: upgrade to 1.42.1 (#1292)

2021-09-14 19:11:39 -07:00

sighup_windows.go

Added repository user authorization support + server flag refactoring + refresh (#890)

2021-03-18 23:03:27 -07:00

storage_azure.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_b2.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_filesystem.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_gcs.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_providers.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_rclone.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_s3.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_sftp_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

storage_sftp.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_webdav.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

suite_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

update_check.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00