Jarek Kowalski 2cb05f7501 logging: added log rotation and improved predictability of log sweep (#1562) ...

* logging: added log rotation and improved predictability of log sweep

With this change logs will be rotated every 50 MB, which prevents
accumulation of giant files while server is running.

This change will also guarantee that log sweep completes at least once
before each invocation of Kopia finishes. Previously it was a goroutine
that was not monitored for completion.

Flags can be used to override default behaviors:

* `--max-log-file-segment-size`
* `--no-wait-for-log-sweep` - disables waiting for full log sweep

Fixes #1561

* logging: added --log-dir-max-total-size-mb flag

This limits the total size of all logs in a directory to 1 GB.

2021-12-03 16:43:46 -08:00

..

app.go

logging: added log rotation and improved predictability of log sweep (#1562)

2021-12-03 16:43:46 -08:00

auto_upgrade.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

cli_progress.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_acl_add.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_acl_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_enable.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_acl_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_acl.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_benchmark_compression.go

cli: benchmark compressor memory usage (#1416)

2021-10-18 21:39:00 -07:00

command_benchmark_crypto.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

command_benchmark_splitters.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_benchmark_test.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_benchmark.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_blob_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_gc.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_blob_list.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

command_blob_shards_modify_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_blob_shards_modify.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_blob_shards.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_blob_show_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_blob_show.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

command_blob_stats.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_blob.go

blob: support for custom blob store sharding (#1299)

2021-09-19 18:50:38 -07:00

command_cache_clear.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_cache_info.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_cache_set_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_cache_set.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_cache_sync_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_cache_sync.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_cache.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_content_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_content_list.go

Support for content-level compression (#1076)

2021-05-22 05:35:27 -07:00

command_content_range_flags.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_content_rewrite.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_content_show.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_content_stats.go

upgraded linter to 1.41.0 (#1144)

2021-06-16 19:44:55 -07:00

command_content_verify_test.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_content_verify.go

Fix endurance test (#1254)

2021-08-27 04:22:18 -07:00

command_content.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_diff.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_index_epoch_list.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_epoch.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_inspect_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_index_inspect.go

cli: added 'index inspect --parallel' flag (#1404)

2021-10-17 18:35:21 -07:00

command_index_list.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_index_optimize.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_index_recover.go

cli: optimized 'kopia index recover' by leveraging partial read and parallelism (#1094)

2021-05-23 22:18:31 -07:00

command_index.go

Implemented epoch-based index manager (#1174)

2021-07-06 21:38:08 -07:00

command_logs_cleanup.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_list.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_logs_session.go

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

command_logs_show.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_logs_test.go

maintenance: send logs to content log as well (#1496)

2021-11-06 23:08:00 -07:00

command_logs.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_ls.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_maintenance_info_test.go

cli: include parameters in maintenance info JSON output (#981)

2021-09-10 17:51:55 -07:00

command_maintenance_info.go

clock: discard monotonic clock component in clock.Now() (#1437)

2021-10-22 15:35:09 -07:00

command_maintenance_run.go

Maintenance improvements for epoch-based index structures (#1225)

2021-08-02 21:08:54 -07:00

command_maintenance_set.go

Added content manager internal logging (#1116)

2021-06-05 08:48:43 -07:00

command_maintenance.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_manifest_delete.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_manifest_show.go

Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)

2021-05-11 22:26:28 -07:00

command_manifest.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_mount.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_policy_edit.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_policy_ls.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_policy_remove.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_actions.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

command_policy_set_compression.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_error_handling.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_files.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_logging_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_logging.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_retention.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_policy_set_scheduling.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_set.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_policy_show.go

policy: revamped policy merge mechanism (#1538)

2021-11-27 18:14:45 -08:00

command_policy.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_repository_change_password_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

command_repository_change_password.go

Repository password change support (#1197)

2021-07-17 07:58:02 -07:00

command_repository_connect_from_config.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_repository_connect_server.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_connect.go

cache: prevent metadata cache thrashing if working set exceeds max defined size (#1557)

2021-12-03 15:35:01 -08:00

command_repository_create.go

policy: revamped policy merge mechanism (#1538)

2021-11-27 18:14:45 -08:00

command_repository_disconnect.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_repository_repair.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

command_repository_set_client.go

repo: limit the duration of kopia.repository caching to 15 minutes (#1196)

2021-07-15 11:32:55 -07:00

command_repository_set_parameters_test.go

content: fixed repo upgrade version (#1286)

2021-09-10 22:51:51 -07:00

command_repository_set_parameters.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_repository_status.go

fixed directory read race condition (#1489)

2021-11-05 10:18:03 -07:00

command_repository_sync.go

Change Mbit/s units to MB/s (base-10) (#1522)

2021-11-18 15:41:40 -08:00

command_repository_validate_provider.go

Added 'kopia repository validate-provider` (#1205)

2021-07-19 21:42:24 -07:00

command_repository.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

command_restore.go

[snapshot restore] use non-atomic writes (#1534)

2021-11-26 13:10:44 -08:00

command_server_cancel.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_flush.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_pause.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_refresh.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_resume.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_start.go

server: maintenance in newly-created repo (#1494)

2021-11-06 17:10:53 -07:00

command_server_status.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_server_tls.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_server_upload.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_server.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

command_session_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_session.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_show.go

minor spelling and grammar fixes (#1318)

2021-09-28 15:19:02 -07:00

command_snapshot_copy_move_history.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_create.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_snapshot_delete.go

cli: added 'snapshot delete --all-snapshots-for-source' (#1397)

2021-10-17 09:46:44 -07:00

command_snapshot_estimate_test.go

Added support for logging policies (#1472)

2021-11-06 10:06:05 -07:00

command_snapshot_estimate.go

cli: test + misc improvements (#1083)

2021-05-17 21:47:11 -07:00

command_snapshot_expire.go

Upgraded linter to 1.40.1 (#1072)

2021-05-15 12:12:34 -07:00

command_snapshot_gc.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_list.go

policy: revamped policy merge mechanism (#1538)

2021-11-27 18:14:45 -08:00

command_snapshot_migrate.go

cli: when migrating snapshot honor destination policies (#1433)

2021-10-21 22:12:06 -07:00

command_snapshot_pin_test.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_snapshot_pin.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_snapshot_restore.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_snapshot_verify.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

command_snapshot.go

cli: added 'snapshot pin' command (#1528)

2021-11-20 20:53:25 -08:00

command_user_add_set.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_delete.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

command_user_info.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user_list.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

command_user.go

cli: major refactoring (#1046)

2021-05-03 10:28:00 -07:00

config.go

introduced structural debug logging and optional JSON output (#1475)

2021-11-03 21:57:37 -07:00

inproc.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00

json_output.go

cli: plumbed through 'textOutput' which controls stdout/stderr writers (#1053)

2021-05-06 20:26:35 -07:00

memory_tracking.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

password_darwin.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

password_linux.go

cli: linux: read KOPIA_USE_KEYRING env (#1337)

2021-10-01 22:56:50 -07:00

password_other.go

linter: upgrade to 1.42.1 (#1292)

2021-09-14 19:11:39 -07:00

password_windows.go

repo: refactored password persistence (#1065)

2021-05-11 21:53:36 -07:00

password.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

profile.go

huge: replaced the use of allocated byte slices with populating gather.WriteBuffer in the repository (#1244)

2021-08-20 08:45:10 -07:00

show_utils.go

refactor: move from io/ioutil to io and os package (#1360)

2021-10-06 08:39:10 -07:00

sighup_unix.go

linter: upgrade to 1.42.1 (#1292)

2021-09-14 19:11:39 -07:00

sighup_windows.go

Added repository user authorization support + server flag refactoring + refresh (#890)

2021-03-18 23:03:27 -07:00

storage_azure.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_b2.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_filesystem.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_gcs.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_providers.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_rclone.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_s3.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_sftp_test.go

blob: changed default shards from {3,3} to {1,3} (#1513)

2021-11-16 06:02:04 -08:00

storage_sftp.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

storage_webdav.go

throtting: implemented a Throttler based on token bucket and configur… (#1512)

2021-11-16 07:39:26 -08:00

suite_test.go

Consolidated format version flags (#1284)

2021-09-08 18:44:03 -07:00

update_check.go

Upgraded linter to 1.43.0 (#1505)

2021-11-11 17:03:11 -08:00