* Add multi-language support
* Adding a guidline to outline the localisation process.
* Moved doc to separate pr
---------
Co-authored-by: lupusA <lupuapps@gmail.com>
* feat(snapshots): added ability to use cron expressions to schedule snapshots
We use `github.com/hashicorp/cronexpr` to parse and evaluate expressions,
as documented in https://github.com/hashicorp/cronexpr#implementation
* upgrade ui
* pr feedback
* chore(ci): upgraded linter to 1.53.3
This flagged a bunch of unused parameters, so the PR is larger than
usual, but 99% mechanical.
* separate lint CI task
* run Lint in separate CI
* Update display on repository summary
* Apply throughout app
* Situate units_test
* Update Command Line documentation
* Envar cleanup
* Rename to BytesString
* Restore envar string available for test
* Remove extraneous empty check and restore UIPreferences field for frontend
* PR: config bool cleanup and missed `BaseEnv`s
* Fix lint and test
Some compression algorithms are not recommended because they
allocate disproportionate amounts of memory. They are still
possible to use, just marked as NOT RECOMMENDED in the UI.
- removed a bunch of hacks and should improve the logging
performance by avoiding interfaces and data translation. This will
allow using of de-sugared loggers in performance-critical
logging situations.
- this will also allow using features of ZAP more directly without
having to reimplement them.
- moved logging.Printf() to testlogging
- refactored `uitask` to store logs in a structural format and
present them as JSON only in the UI
- renamed printf_logger.go to printf.go so that fewer columns are used
in the logs
This allows KopiaUI server to start when the repository directory
is not mounted or otherwise unavailable. Connection attempts will
be retried indefinitely and user will see new `Initializing` page.
This also exposes `Open` and `Connect` as tasks allowing the user to see
logs directly in the UI and cancel the operation.
* cli: refactored snapshot list
* cli: show range tags in snapshot list
For example if N snapshots are coalesced together because they
have identical roots we may emit now:
```
2021-03-31 23:09:27 PDT ked3400debc7dd61baffab070bafd59cd (monthly-10)
2021-04-30 06:12:53 PDT kd0576d212e55a831b7ff1636f90a7233 (monthly-4..9)
+ 5 identical snapshots until 2021-09-30 23:00:19 PDT
2021-10-31 23:22:25 PDT k846bf22aa2863d27f05e820f840b14f8 (monthly-3)
2021-11-08 21:29:31 PST k5793ddcd61ef27b93c75ab74a5828176 (latest-1..3,hourly-1..13,daily-1..7,weekly-1..4,monthly-1..2,annual-1)
+ 18 identical snapshots until 2021-12-04 10:09:54 PST
```
* server: server-side coalescing of snapshot
* ui: added coalescing of retention tags
* ui: changed how PolicyEditor is instantiated via a route
* server: added paths/resolve API
* server: refresh affected source manager after policy change
Also switched 15-second refresh cycle which is way too aggressive
to 30-minute cycle (manual refresh button can be used if needed).
* policy: allow overriding top-level policy for estimation
* server: changed source create API to always require policy
* ui: streamlined new snapshot and estimate flow
* linter fix
* cli: added a flag to create repository with v2 index features
* content: plumb through compression.ID parameter to content.Manager.WriteContent()
* content: expose content.Manager.SupportsContentCompression
This allows object manager to decide whether to create compressed object
or let the content manager do it.
* object: if compression is requested and the repo supports it, pass compression ID to the content manager
* cli: show compression status in 'repository status'
* cli: output compression information in 'content list' and 'content stats'
* content: compression and decompression support
* content: unit tests for compression
* object: compression tests
* testing: added integration tests against v2 index
* testing: run all e2e tests with and without content-level compression
* htmlui: added UI for specifying index format on creation
* cli: additional tests for 'content ls' and 'content stats'
* applied pr suggestions
* cli: Added --max-examples-per-bucket flag to 'kopia snapshot estimate'
Added and cleaned up a bunch of unit tests.
Fixes#1054
* cli: misc tests to increase code coverage of the cli package
* ci: move code coverage run into separate GH job
* uitask: added support for reporting string progress info
* server: report current directory as task progress
* snapshot: created reusable Estimate() method to be used during upload, cli estimate and via API
* cli: switched to snapshotfs.Estimate()
* server: added API to estimate snapshot size
* kopia-ui: fixed directory selector
* htmlui: streamlined new snapshot flow and cleaned up policy setting
See https://youtu.be/8p6csuoB3kg
* repo: refactored connect code set up cache for server repositories
- improved logic to close the cache on last connection
- preemptively add all contents with a prefix to the cache
- refactored how config is loaded and saved
Now cache dir will be stored as relative and resolved to absolute as
part of loading and saving the file, in all other places cache dir
is expected to be absolute.
* server: removed cache directory from the API and UI
This won't be easily available and does not seem useful to expose
anyway.
* cli: enabled cache commands for server repositories
* cli: added KOPIA_CACHE_DIRECTORY environment variable
This is used on two occassions - when setting up connection (it gets
persisted in the config) and later when opening (to override the
cache location from config). It makes setting up docker container with
mounted cache somewhat easier with one environment variable.
* cli: show cache size for the server cache
* tls: present more helpful error message that includes SHA256 fingerprint of the TLS server on mismatch
* server: return the name of user who attempted to login when authentication fails
* uitask: added package for managing and introspection into tasks running inside the process
* server: added API for getting details of tasks running inside the server
* htmlui: added new tab called 'Tasks'
This allows access to progres, logs and cancelation for long-running
tasks (Snapshots, Maintenance, and in the future Restore, Estimate,
Verify)
* snapshot: improve counters returned from the upload
* server: reworked authn/authz
Previously authentication was done as an wrapper handler and
authorization was inlined. This change moves authn/authz handlers
inside the server and implements separate authorization module that's
individually tested.
Also fixed an issue where server users were not able to see global
or host-level policies.
* PR feedback
* cli: added --tls-print-server-cert flag
This prints complete server certificate that is base64 and PEM-encoded.
It is needed for Electron to securely connect to the server outside of
the browser, since there's no way to trust certificate by fingerprint.
* server: added repo/exists API
* server: added ClientOptions to create and connect API
* server: exposed current-user API
* server: API to change description of a repository
* htmlui: refactored connect/create flow
This cleaned up the code a lot and made UX more obvious.
* kopia-ui: simplified repository management UX
Removed repository configuration window which was confusing due to
the notion of 'server'.
Now KopiaUI will automatically launch 'kopia server --ui' for each
config found in the kopia config directory and shut it down every
time repository is disconnected.
See https://youtu.be/P4Ll_LR4UVM for a quick demo.
Fixes#583
* repo: refactored client-specific options (hostname,username,description,readonly) into new struct that is JSON-compatible with current config
* cli: added 'repository set-client' to configure parameters of connected repository
* cli: cleaned up 'repository status' output
* cli: simplified mount command
See https://youtu.be/1Nt_HIl-NWQ
It will always use WebDAV on Windows and FUSE on Unix. Removed
confusing options.
New usage:
$ kopia mount [--browse]
Mounts all snapshots in a temporary filesystem directory
(both Unix and Windows).
$ kopia mount <object> [--browse]
Mounts given object in a temporary filesystem directory
(both Unix and Windows).
$ kopia mount <object> z: [--browse]
Mounts given object as a given drive letter in Windows (using
temporary WebDAV mount).
$ kopia mount <object> * [--browse]
Mounts given object as a random drive letter in Windows.
$ kopia mount <object> /mount/path [--browse]
Mounts given object in given path in Unix.
<object> can be the ID of a directory 'k<hash>' or 'all'
Optional --browse automatically opens OS-native file browser.
* htmlui: added UI for mounting directories
See https://youtu.be/T-9SshVa1d8 for a quick demo.
Also replaced some UI text with icons.
* lint: windows-specific fix
* kopia-ui: added ability to connect to kopia server
* kopia-ui: update status page to show some data for repositories connected to API server
* kopia-ui: hide user@host selection dropdown for kopia server repositories
Percentage based on last-known snapshot size
* server: exposed last completed snapshot size in the API
* cli: added support for progress indicator (percentage based on last-known snapshot size)
* htmlui: added progress indicator in the UI (percentage based on last-known snapshot size)
CreateSnapshotSource API for ensuring source exists
Upload - starts upload on a given source or matching sources
Cancel - cancels upload on a given source or matching sources
/api/v1/repo/create
/api/v1/repo/connect
/api/v1/repo/disconnect
Refactored server code and fixed a number of outstanding robustness
issues. Tweaked the API responses a bit to make more sense when consumed
by the UI.
Also introduced strongly typed content.ID and manifest.ID (instead of string)
This aligns identifiers across all layers of repository:
blob.ID
content.ID
object.ID
manifest.ID
