Commit Graph

12 Commits

Author SHA1 Message Date
Jarek Kowalski
4b4628a21e Repository maintenance support (#411)
Maintenance: support for automatic GC

Moved maintenance algorithms from 'cli' to 'repo/maintenance' package

Added support for CLI commands:

kopia gc - performs quick maintenance
kopia gc --full- perform full maintenance

Full maintenance performs snapshot gc, but it's not safe to do this automatically possibly in parallel to snapshots being taken. This will be addressed ~0.7 timeframe.
2020-04-14 00:11:41 -07:00
Jarek Kowalski
70d4c8764a cli: improvements to content selection for list/rewrite/stats/verify (#409)
They now uniformly support 3 flags:

--prefix=P       selects contents with the specified prefix
--prefixed       selects contents with ANY prefix
--non-prefixed   selects non-prefixed contents

Also changed content manager iteration API to support ranges.

cli: add --prefix to 'blob gc' and 'blob stats'
2020-04-06 18:43:41 -07:00
Jarek Kowalski
6cb9b8fa4f repo: refactored public API (#318)
* This is 99% mechanical:

Extracted repo.Repository interface that only exposes high-level object and manifest management methods, but not blob nor content management.

Renamed old *repo.Repository to *repo.DirectRepository

Reviewed codebase to only depend on repo.Repository as much as possible, but added way for low-level CLI commands to use DirectRepository.

* PR fixes
2020-03-26 08:04:01 -07:00
Jarek Kowalski
6f68b726a7 cli: improved 'content rewrite'
- removed confusing '--prefixed' option
- print timestamp of contents as they are rewritten
2020-03-05 17:33:56 -08:00
Jarek Kowalski
c8fcae93aa logging: refactored logging
This is mostly mechanical and changes how loggers are instantiated.

Logger is now associated with a context, passed around all methods,
(most methods had ctx, but had to add it in a few missing places).

By default Kopia does not produce any logs, but it can be overridden,
either locally for a nested context, by calling

ctx = logging.WithLogger(ctx, newLoggerFunc)

To override logs globally, call logging.SetDefaultLogger(newLoggerFunc)

This refactoring allowed removing dependency from Kopia repo
and go-logging library (the CLI still uses it, though).

It is now also possible to have all test methods emit logs using
t.Logf() so that they show up in failure reports, which should make
debugging of test failures suck less.
2020-02-25 17:24:44 -08:00
Jarek Kowalski
ac70a38101 lint: upgraded to 1.22.2 and make lint issues a build failure
fixed or silenced linter warnings, mostly due to magic numeric constants
2020-01-03 16:39:30 -08:00
Jarek Kowalski
6217df1a87 lint: switched to 1.21 and fixed a ton of whitespace issues discovered
by new wsl linter
2019-11-26 06:49:49 -08:00
Jarek Kowalski
b365d3414c removed content.Manager.ListContents*() API 2019-07-27 19:10:58 -07:00
Jarek Kowalski
6f5abf5e09 added content.Manager Iterate()
This uses callbacks avoids having to buffer
all content infos in memory and does not require holding a lock.
2019-07-26 21:31:25 -07:00
Jarek Kowalski
829faf8e1a added 'rewrite --prefixed' flag to only rewrite prefixed content IDs (by default only rewrites non-prefixed IDs) 2019-07-18 20:29:27 -10:00
Jarek Kowalski
72520029b0 golangci-lint: added more linters
Also fixed pre-existing lint errors.
2019-06-02 22:56:57 -07:00
Jarek Kowalski
54edb97b3a refactoring: renamed repo/block to repo/content
Also introduced strongly typed content.ID and manifest.ID (instead of string)

This aligns identifiers across all layers of repository:

blob.ID
content.ID
object.ID
manifest.ID
2019-06-01 22:24:19 -07:00