Commit Graph

3 Commits

Author SHA1 Message Date
Shikhar Mall
b592776edf feat(repository): persistence for blob-retention configuration (#1596)
* feat: persisting retention options in repository blob

 - plumb retention parameters through wrapped storage
 - generalize aes encryption mechanism
 - rewrite the retention blob on password change
 - do not write retention blob when empty

* handle retention-blob not-found failures

* cli params to set retention modes on repository create

* enable versioned map mock storage with retention settings

* adding unit tests

* write format and retention blob with retention settings if available

* rename certain functions and constants specific to format blob

* delete retention cache on password-change

* fix: replace SetTime() api call with TouchBlob()

* Update repo/repository_test.go

Co-authored-by: Nick <nick@kasten.io>

* pr feedback and codecov improvements

* fix: rename retention-blob structures to generic blob-cfg

* fix: remove minio dependency on retention constants

Co-authored-by: Shikhar Mall <shikhar@kasten.io>
Co-authored-by: Nick <nick@kasten.io>
2022-01-22 08:37:00 -08:00
Jarek Kowalski
7401684e71 blob: replaced blob.Storage.SetTime() method with blob.PutOptions.SetTime (#1595)
* sharded: plumbed through blob.PutOptions

* blob: removed blob.Storage.SetTime() method

This was only used for `kopia repo sync-to` and got replaced with
an equivalent blob.PutOptions.SetTime, which wehn set to non-zero time
will attempt to set the modification time on a file.

Since some providers don't support changing modification time, we
are able to emulate it using per-blob metadata (on B2, Azure and GCS),
sadly S3 is still unsupported, because it does not support returning
metadata in list results.

Also added PutOptions.GetTime, which when set to not nil, will
populate the provided variable with actual time that got assigned
to the blob.

Added tests that verify that each provider supports GetTime
and SetTime according to this spec.

* blob: additional test coverage for filesystem storage

* blob: added PutBlobAndGetMetadata() helper and used where appropriate

* fixed test failures

* pr feedback

* Update repo/blob/azure/azure_storage.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* Update repo/blob/filesystem/filesystem_storage.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* Update repo/blob/filesystem/filesystem_storage.go

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>

* blobtesting: fixed object_locking_map.go

* blobtesting: removed SetTime from ObjectLockingMap

Co-authored-by: Shikhar Mall <mall.shikhar.in@gmail.com>
2021-12-18 14:00:20 -08:00
Shikhar Mall
3f4dab2202 versioned map mock storage for blob-retention unit-testing (#1590)
* versioned map mock storage

* Update internal/blobtesting/versionedmap.go

Co-authored-by: Julio Lopez <julio+gh@kasten.io>

* versionedMapStorage -> objectLockingMap

* move out write checks in a dedicated method

Co-authored-by: Shikhar Mall <shikhar@kasten.io>
Co-authored-by: Julio Lopez <julio+gh@kasten.io>
2021-12-17 13:26:34 -08:00