Jarek Kowalski cbb7d68fb6 feat(repository): implemented format.Manager - take 2 (#2349) ...

* implemented format blob cache abstraction

* moved upgrade lock logic to repo/format

* moved set parameters logic to repo/format

* moved change password functionality to repo/format

* mechanical changes

* mechanical changes to react to format manager interface

* get current repository format bytes instead of static

* implemented format.Manager which dynamically fetches and caches latest format blob

* repo changes to use format.Manager

* fixed failing unit test due to different timings

* reduced lock contention by using RWMutex

* serve immutable parts of format without any locks

* increase test timeout

* fixed handling of negative validDuration

The new rules are:

- validDuration < 0 - ignore initial cached file, refresh every 15min
- validDuration > 15min - refresh every 15 minutes
- validDuration > 0 && validDuration <= 15min - refresh using provided
  interval (mostly used for testing)

2022-09-02 18:55:01 -07:00

..

app.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

auto_upgrade.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

cli_progress.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_acl_add.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_acl_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_enable.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_acl.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_benchmark_compression.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_benchmark_crypto.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_benchmark_ecc.go

feat(cli): Added ECC related options to repository create cli command (#2308)

2022-08-13 08:49:22 -07:00

command_benchmark_encryption.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_benchmark_hashing.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_benchmark_splitters.go

feat(cli): improved kopia benchmark commands (#1849)

2022-03-26 14:28:08 +00:00

command_benchmark_test.go

feat(cli): improved kopia benchmark commands (#1849)

2022-03-26 14:28:08 +00:00

command_benchmark.go

feat(repository): Error correction for blobs (#2270)

2022-08-09 18:24:25 -07:00

command_blob_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_gc.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_list.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_blob_shards_modify_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_blob_shards_modify.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_blob_shards.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_blob_show_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_blob_show.go

refactor(repository): refactored Crypter to an unexported interface (#2251)

2022-07-30 14:57:56 +00:00

command_blob_stats.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_blob.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_cache_clear.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_cache_info.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_cache_prefetch.go

feat(repository): added 'hint' to Prefetch methods. (#1825)

2022-03-12 23:16:39 -08:00

command_cache_set_test.go

fix(cli): fixed 'cache set' CLI regression due to kingpin change (#2173)

2022-07-10 13:26:41 -07:00

command_cache_set.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_cache_sync_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_cache_sync.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_cache.go

feat(repository): added PrefetchObjects() API (#1779)

2022-03-06 14:30:58 -08:00

command_content_delete.go

refactor(repository): ensure we always parse content.ID and object.ID (#1960)

2022-05-25 14:15:56 +00:00

command_content_list.go

Support for content-level compression (#1076)

2021-05-22 05:35:27 -07:00

command_content_range_flags.go

refactor(repository): ensure we always parse content.ID and object.ID (#1960)

2022-05-25 14:15:56 +00:00

command_content_rewrite.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_content_show.go

refactor(repository): ensure we always parse content.ID and object.ID (#1960)

2022-05-25 14:15:56 +00:00

command_content_stats.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_content_verify_test.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_content_verify.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_content.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_diff.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_index_epoch_list.go

refactor(repository): ensure MutableParameters are never cached (#2284)

2022-08-06 18:11:32 -07:00

command_index_epoch.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_inspect_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_index_inspect.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_index_list.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_optimize.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_index_recover.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_index.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_logs_cleanup.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_list.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_session.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_logs_show.go

refactor(repository): refactored Crypter to an unexported interface (#2251)

2022-07-30 14:57:56 +00:00

command_logs_test.go

maintenance: send logs to content log as well (#1496)

2021-11-06 23:08:00 -07:00

command_logs.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_ls.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_maintenance_info_test.go

cli: include parameters in maintenance info JSON output (#981)

2021-09-10 17:51:55 -07:00

command_maintenance_info.go

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

command_maintenance_run.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_maintenance_set.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_maintenance.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_manifest_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_manifest_show.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_mount.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_policy_edit.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_policy_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_policy_remove.go

fix(cli): fixed and unified help text for policy commands (#1829)

2022-03-16 00:46:57 -07:00

command_policy_set_actions.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

command_policy_set_compression.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_error_handling.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_files.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_logging_test.go

feat(cli): improved formatting of 'policy show' outputs (#1767)

2022-02-22 22:21:48 -08:00

command_policy_set_logging.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_retention.go

feat(snapshots): add option to ignore empty snapshots being saved (#2036)

2022-06-13 03:41:08 +00:00

command_policy_set_scheduling.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_upload_test.go

feat(snapshots): improved performance when uploading huge files (#2064)

2022-06-24 07:38:07 +00:00

command_policy_set_upload.go

feat(snapshots): improved performance when uploading huge files (#2064)

2022-06-24 07:38:07 +00:00

command_policy_set.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_policy_show.go

feat(snapshots): improved performance when uploading huge files (#2064)

2022-06-24 07:38:07 +00:00

command_policy.go

fix(cli): fixed and unified help text for policy commands (#1829)

2022-03-16 00:46:57 -07:00

command_repo_status_test.go

feat(cli): add --json output to 'repo status' command (#1834)

2022-03-17 22:22:24 -07:00

command_repo_throttle_test.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_repository_change_password_test.go

refactor(repository): moved format blob management to separate package (#2245)

2022-07-30 14:13:52 -07:00

command_repository_change_password.go

feat(repository): implemented format.Manager - take 2 (#2349)

2022-09-02 18:55:01 -07:00

command_repository_connect_from_config.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_repository_connect_server.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_connect.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_repository_create.go

chore(cli): make ecc options visible but marked as experimental (#2322)

2022-08-19 15:04:40 +00:00

command_repository_disconnect.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_repair.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_repository_set_client.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_repository_set_parameters_test.go

refactor(repository): moved format blob management to separate package (#2245)

2022-07-30 14:13:52 -07:00

command_repository_set_parameters.go

feat(repository): implemented format.Manager - take 2 (#2349)

2022-09-02 18:55:01 -07:00

command_repository_status.go

feat(repository): implemented format.Manager - take 2 (#2349)

2022-09-02 18:55:01 -07:00

command_repository_sync.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_repository_throttle_get.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_repository_throttle_set.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_repository_throttle.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_repository_upgrade_test.go

feat(cli): flag to allow placing the upgrade lock without doing anything else (#2329)

2022-08-19 13:48:08 -07:00

command_repository_upgrade.go

feat(repository): implemented format.Manager - take 2 (#2349)

2022-09-02 18:55:01 -07:00

command_repository_validate_provider.go

fix(providers): fixed DoNotRecreate and tests for gcs (#1688)

2022-01-29 09:12:07 -08:00

command_repository.go

feat(cli): New Upgrade CLI / Switch to Format Version 3 (upgrade coordination) (#1818)

2022-07-27 16:23:45 -07:00

command_restore.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_server_cancel.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_control_test.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_server_flush.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_server_pause.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_refresh.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_server_resume.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_shutdown.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_server_snapshot.go

server: support for server control APIs and tooling (#1644)

2022-01-03 18:48:38 -08:00

command_server_source_manager_action.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_server_start.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_server_status.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_server_throttle_get.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_server_throttle_set.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_server_throttle.go

feat(cli): added CLI for controlling throttler (#1956)

2022-05-18 01:27:06 -07:00

command_server_tls.go

chore(general): updated linter to 1.44.0 (#1681)

2022-01-25 21:21:13 -08:00

command_server.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

command_session_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_session.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_show.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_snapshot_copy_move_history.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_create.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_delete.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_estimate_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_snapshot_estimate.go

feat(snapshots): Allow users to dynamically create entries in a directory during an upload (#1996)

2022-06-14 19:08:49 -07:00

command_snapshot_expire.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_fix_invalid_files.go

feat(cli): implementation for 'kopia snapshot fix' (#1930)

2022-05-25 01:17:55 +00:00

command_snapshot_fix_remove_files.go

refactor(repository): ensure we always parse content.ID and object.ID (#1960)

2022-05-25 14:15:56 +00:00

command_snapshot_fix_test.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

command_snapshot_fix.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_gc.go

refactor(cli): deprecate snapshot gc command (#1973)

2022-05-26 05:51:36 +00:00

command_snapshot_list_test.go

fix(cli): fixed snapshot sizes in the snapshot list (#2148)

2022-07-06 05:54:27 +00:00

command_snapshot_list.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_migrate.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_pin_test.go

fix(cli): add retention to JSON output (#1992)

2022-05-31 13:43:42 -07:00

command_snapshot_pin.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot_restore.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_storage_stats_test.go

fix(cli): add retention to JSON output (#1992)

2022-05-31 13:43:42 -07:00

command_snapshot_verify.go

feat(repository): introduced fs.UTCTimestamp (#2343)

2022-09-02 10:35:59 -07:00

command_snapshot.go

feat(cli): implementation for 'kopia snapshot fix' (#1930)

2022-05-25 01:17:55 +00:00

command_user_add_set.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_user_info.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

config.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

inproc.go

feat(repository): added required features to the repository (#2220)

2022-07-29 09:31:17 -07:00

json_output.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

observability_flags_test.go

feat(cli): added open telemetry tracing support (#1988)

2022-05-28 10:39:00 -07:00

observability_flags.go

fix(cli): Fix /debug/pprof/heap returning 404 (#2339)

2022-08-23 08:55:25 +00:00

password_darwin.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

password_linux.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

password_other.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

password_windows.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

password.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

profile.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

show_utils.go

feat(cli): improved formatting of 'policy show' outputs (#1767)

2022-02-22 22:21:48 -08:00

sighup_unix.go

linter: upgrade to 1.42.1 (#1292)

2021-09-14 19:11:39 -07:00

sighup_windows.go

Added repository user authorization support + server flag refactoring + refresh (#890)

2021-03-18 23:03:27 -07:00

storage_azure.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_b2.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_filesystem.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_gcs.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_gdrive.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_providers.go

test(infra): improved support for in-process testing (#2169)

2022-07-09 18:22:50 -07:00

storage_rclone.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_s3.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_sftp_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

storage_sftp.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

storage_webdav.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

suite_test.go

refactor(repository): moved format blob management to separate package (#2245)

2022-07-30 14:13:52 -07:00

throttle_get.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

throttle_set.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00

update_check.go

chore(ci): upgraded linter to 1.48.0 (#2294)

2022-08-09 06:07:54 +00:00