a8a772c293
* performance: improve performance of fragmented index lookups In a typical repository there will be many small indexes and few large ones. If the number gets above ~100 or so, things get very slow. It helps to pre-merge very small indexes in memory to reduce the number of binary searches and mmaped IO to perform. In some extreme cases where we have many uncompacted index segments (each with just 1-2 entries), the savings are as quite dramatic. In one case with >100 index segments the time to run `kopia snapshot verify` went from 10m to 0.5m after this change. In another well-maintained repository with 1.2M contents and about 25 segments, the time to run `kopia snapshot verify` went from 48s to 35s. Co-authored-by: Julio López <julio+gh@kasten.io>
Kopia
n.
Kopia is a simple, cross-platform tool for managing encrypted backups in the cloud. It provides fast, incremental backups, secure, client-side end-to-end encryption, compression and data deduplication.
Unlike other cloud backup solutions, the user is in full control of the backup storage and responsible for purchasing one of the cloud storage products (such as Google Cloud Storage), which offer great durability and availability for the data.
Kopia in action
Using kopia command line tool:
Kopia UI - experimental user interface
Getting Started
See Documentation for more information.
Building Kopia
See Build Infrastructure for more information on building Kopia and working with the source code.
Licensing
Kopia is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Contribution Guidelines
Kopia is open source and contributions are welcome. For more information on how to contribute see the Contribution Guidelines.
Reporting Security Issues
If you find a security issue you'd like to disclose privately, please contact kopia-pmc@googlegroups.com or via direct message to maintainers on Slack.
Disclaimer
Kopia is a personal project and is not affiliated with, supported or endorsed by Google.
Cryptography Notice
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with symmetric algorithms. The form and manner of this distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
