non-optimized (0.5.0)
0. BLAKE2B-256-128 AES256-GCM-HMAC-SHA256 644.9 MiB / second
before this change:
0. BLAKE2B-256-128 AES256-GCM-HMAC-SHA256 655.9 MiB / second
after (this change):
0. BLAKE2B-256-128 AES256-GCM-HMAC-SHA256 781.5 MiB / second
* performance: plumbed through output buffer to encryption and hashing, so that the caller can pre-allocate/reuse it
* testing: fixed how we do comparison of byte slices to account for possible nils, which can be returned from encryption
New ciphers are using authenticated encryption with associated data
(AEAD) and per-content key derived using HMAC-SHA256:
* AES256-GCM-HMAC-SHA256
* CHACHA20-POLY1305-HMAC-SHA256
They support content IDs of arbitrary length and are quite fast:
On my 2019 MBP:
- BLAKE2B-256 + AES256-GCM-HMAC-SHA256 - 648.7 MiB / second
- BLAKE2B-256 + CHACHA20-POLY1305-HMAC-SHA256 - 597.1 MiB / second
- HMAC-SHA256 + AES256-GCM-HMAC-SHA256 351 MiB / second
- HMAC-SHA256 + CHACHA20-POLY1305-HMAC-SHA256 316.2 MiB / second
Previous ciphers had several subtle issues:
* SALSA20 encryption, used weak nonce (64 bit prefix of content ID),
which means that for any two contents, whose IDs that have the same
64-bit prefix, their plaintext can be decoded from the ciphertext
alone.
* AES-{128,192,256}-CTR were not authenticated, so we were
required to hash plaintext after decryption to validate. This is not
recommended due to possibility of subtle timing attacks if an attacker
controls the ciphertext.
* SALSA20-HMAC was only validating checksum and not that the ciphertext
was for the correct content ID.
New repositories cannot be created using deprecated ciphers, but they
will still be supported for existing repositories, until at least 0.6.0.
The users are encouraged to migrate to one of new ciphers when 0.5.0 is
out.
This is mostly mechanical and changes how loggers are instantiated.
Logger is now associated with a context, passed around all methods,
(most methods had ctx, but had to add it in a few missing places).
By default Kopia does not produce any logs, but it can be overridden,
either locally for a nested context, by calling
ctx = logging.WithLogger(ctx, newLoggerFunc)
To override logs globally, call logging.SetDefaultLogger(newLoggerFunc)
This refactoring allowed removing dependency from Kopia repo
and go-logging library (the CLI still uses it, though).
It is now also possible to have all test methods emit logs using
t.Logf() so that they show up in failure reports, which should make
debugging of test failures suck less.
Also introduced strongly typed content.ID and manifest.ID (instead of string)
This aligns identifiers across all layers of repository:
blob.ID
content.ID
object.ID
manifest.ID
The splitter in question was depending on
github.com/silvasur/buzhash which is not licensed according to FOSSA bot
Switched to new faster implementation of buzhash, which is
unfortunately incompatible and will split the objects in different
places.
This change is be semi-breaking - old repositories can be read, but
when uploading large objects they will be re-uploaded where previously
they would be de-duped.
Also added 'benchmark splitters' subcommand and moved 'block cryptobenchmark'
subcommand to 'benchmark crypto'.
