Commit Graph

17 Commits

Author SHA1 Message Date
Aaron Alpar
4c0fe390e1 feat(repository): new repository upgrade sub-command to validate indices. (#2561)
* remove unused context param

* add index validation

* add test

* add command stub

* add comments. fixup.

* Update cli/command_repository_upgrade.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* fixup bad merge

* add validation to command line

* set-parameter fixup

* move poison blob to commit

* fixups from PR comments

* fix bug for set-parameter

* robust commit-mode. add upgrade test

* fixup incorrect methods

* fixup for lint

* fixup for lint

* fixup named return values

* fixup failing test

* fixup lint

* fixup for lint

* Update repo/content/committed_read_manager.go

Co-authored-by: Nick <nick@kasten.io>

* Update repo/content/committed_read_manager.go

Co-authored-by: Nick <nick@kasten.io>

* use xor instead of fixed value

* lint fixups

* fixup for lint cyclomatic complexity

* fixup bad merge

* fixup set-parameters downgrade

* Update cli/command_repository_upgrade.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* Update cli/command_repository_upgrade.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* fixup set-parameters downgrade

* cleanup set-parameters test

* fixup tests, refactor for index verification

* fixups and clarity

* remove set-parameter downgrade check

* index on feat-k10-12441-B-validate-index: b63879cd Merge 'master' into feat-k10-12441-B-validate-index

* trigger build

* remove needless blank line

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
Co-authored-by: Nick <nick@kasten.io>
2022-12-13 09:42:09 -08:00
Aaron Alpar
ca7df80e64 fix(cli): do not allow index downgrade by using set-parameter (#2629)
* disallow downgrades

* fixup merge
2022-12-06 18:33:55 -08:00
atom
c5efed01f4 feat(cli): Support displaying storage values in base-2 [#2492] (#2502)
* Update display on repository summary

* Apply throughout app

* Situate units_test

* Update Command Line documentation

* Envar cleanup

* Rename to BytesString

* Restore envar string available for test

* Remove extraneous empty check and restore UIPreferences field for frontend

* PR: config bool cleanup and missed `BaseEnv`s

* Fix lint and test
2022-10-24 19:00:36 -07:00
Jarek Kowalski
cbb7d68fb6 feat(repository): implemented format.Manager - take 2 (#2349)
* implemented format blob cache abstraction

* moved upgrade lock logic to repo/format

* moved set parameters logic to repo/format

* moved change password functionality to repo/format

* mechanical changes

* mechanical changes to react to format manager interface

* get current repository format bytes instead of static

* implemented format.Manager which dynamically fetches and caches latest format blob

* repo changes to use format.Manager

* fixed failing unit test due to different timings

* reduced lock contention by using RWMutex

* serve immutable parts of format without any locks

* increase test timeout

* fixed handling of negative validDuration

The new rules are:

- validDuration < 0 - ignore initial cached file, refresh every 15min
- validDuration > 15min - refresh every 15 minutes
- validDuration > 0 && validDuration <= 15min - refresh using provided
  interval (mostly used for testing)
2022-09-02 18:55:01 -07:00
Jarek Kowalski
51dcaa985d chore(ci): upgraded linter to 1.48.0 (#2294)
Mechanically fixed all issues, added `lint-fix` make target.
2022-08-09 06:07:54 +00:00
Jarek Kowalski
6160ee5668 refactor(repository): moved format blob management to separate package (#2245)
* refactor(repository): moved format blob management to separate package

This is completely mechanical, no behavior changes, only:

- moved types and functions to a new package
- adjusted visibility where needed
- added missing godoc
- renamed some identifiers to align with current usage
- mechanically converted some top-level functions into member functions
- fixed some mis-named variables

* refactor(repository): moved content.FormatingOptions to format.ContentFormat
2022-07-30 14:13:52 -07:00
Jarek Kowalski
30456d13e7 refactor(repository): refactored Crypter to an unexported interface (#2251) 2022-07-30 14:57:56 +00:00
Jarek Kowalski
b9be9632a2 feat(repository): added required features to the repository (#2220)
* feat(repository): added `required features` to the repository

This is intended for future compatibility to be able to reliably
stop old kopia client from being able to open a repository when
the old code does not understand new `required feature`.

Required features are checked on startup and periodically using the
same method as upgrade lock, where they will return errors during blob
operations.

* pr feedback
2022-07-29 09:31:17 -07:00
Jarek Kowalski
03da9b1f0e feat(cli): improved safety of v1->v2 index format upgrade (#2223)
When upgrading from legacy to epoch manager-based index, we will write
an intentionally-corrupted index blob, such that old clients won't be
able to understand it when they read the repository index using legacy
format.

The error message emitted by very old clients is not great, but it's
safer to do that rather than corrupt the repository.

Note that this additional safety has a delay of up to 15 minutes
which is the time required for old clients to stop relying on index list
cache in case of very long-running snapshots, server or KopiaUI.
2022-07-28 17:32:31 -07:00
Jarek Kowalski
56f3046d8a refactor(repository): introduce interface for reading FormattingOptions (#2235)
Instead of passing static content.FormattingOptions (and caching it)
we now introduce an interface to provide its values.

This will allow the values to dynamically change at runtime in the
future to support cases like live migration.
2022-07-28 17:27:04 -07:00
Shikhar Mall
63bedd3446 feat(cli): allow changing retention parameters from CLI (#1680)
Co-authored-by: Shikhar Mall <small@kopia.io>
2022-02-02 19:04:22 -08:00
Jarek Kowalski
32ed220a6c build(lint): enabled gochecknoglobals and tagged existing globals (#1664) 2022-01-15 12:54:56 -08:00
cristihcd
d9c1bd0758 minor spelling and grammar fixes (#1318) 2021-09-28 15:19:02 -07:00
Jarek Kowalski
8b2b91f9f9 content: fixed repo upgrade version (#1286)
* content: fixed repo upgrade version

Previously upgrade would enable epoch manager and index v2 but would
not set the version of the format itself. Everything worked fine
but it would not protect from old kopia opening the repository.

* ci: added compatibility test that uses real 0.8 and current binaries
2021-09-10 22:51:51 -07:00
Jarek Kowalski
7e68d8e4c1 Consolidated format version flags (#1284) 2021-09-08 18:44:03 -07:00
Jarek Kowalski
62ad437bb6 Implemented epoch-based index manager (#1174)
* epoch: misc fixes and logging

* blob: misc helpers

* cli: removed useless 'repository upgrade', replaced by 'repository set-parameters'

* content: implemented indexBlobManagerV1 which uses epoch manager

* cli: commands to manipulate repository epoch parameters

* cli: commands to examine epoch-based indexes

* content: added test suite that uses epoch-based index manager

* content: fixed a ton of test data races caused by sharing blobtesting.DataMap

* cli: additional tests and validation for 'repository set-params'

* testing: replaced the use of suite with our own, since suite is not parallelizable
2021-07-06 21:38:08 -07:00
Jarek Kowalski
85cfbb1afd cli: added 'kopia repo set-parameters' to override mutable parameters (#1148) 2021-06-19 18:47:13 -07:00