* crypto: implement more primitives for the MemoryStore to work in tests
* crypto: change the shape of the `CryptoStoreLock` API
In particular:
- make the lock work across multiple threads of the same process trying to acquire it,
using `num_holders`.
- add a mechanism to get the lock only once (for the NSE process, in case the main app
had acquired the lock before).
* client: add a cross-process crypto store lock, enable it with `Encryption::enable_cross_process_store_lock`
* client: make `preshare_room_key` a critical section of the cross-process lock
* sliding sync: make it possible to define different timeouts for a `SlidingSyncInstance`
This will be handy for the NSE process on iOS, which has very little time to wait for the proxy's responses.
* feat: implement the `EncryptionSync` API (renamed from `Notification` API)
* fixup! client: add a cross-process crypto store lock, enable it with `Encryption::enable_cross_process_store_lock`
* feat: allow disabling e2ee / to-device in the RoomList API
* feat: use same SS id for main/NSE process, reload to-device token from disk before each encryption sync
* fix: better error handling if restoring the to-device token failed
* feat: add logs for the locking functions
* test: add a few tests for encryption sync
* feat: add `reload_caches` method in the `EncryptionSync` + FFI bindings
* chore: clean up FFI loop
* encryption sync: Remove unused errors, specialize some errors
* feat: include termination reason in the encryption sync loop
* feat: add more logs
* chore: fmt + clippy + doc
* comment: precise only in the presence of another process
* Tweak `room_list` APIs to include `_with_encryption` variants
* chore: rustfmt
This implements a value-based lock in the crypto stores. The intent is to use that for multiple processes to be able to make writes into the store concurrently, while still cooperating on who does them. In particular, we need this for #1928, since we may have up to two different processes trying to write into the crypto store at the same time.
## New methods in the `CryptoStore` trait
The idea is to introduce two new methods touching **custom values** in the crypto store:
- one to atomically insert a value, only if it was missing (so, not following the semantics of `upsert` used in the `set_custom_value`)
- one to atomically remove a custom value
Those two operations match the semantics we want:
- take the lock only if it ain't taken already == insert an entry only if it was missing
- release the lock = remove the entry
By looking at the number of lines affected by the query, we can infer whether the insert/remove happened or not, that is, if we managed to take the lock or not.
## High-level APIs
I've also added an high-level API, `CryptoStoreLock`, that helps managing such a lock, and adds some niceties on top of that:
- exponential backoff to retry attempts at acquiring the lock, when it was already taken
- attempt to gracefully recover when the lock has been taken by an app that's been killed by the environment
- full configuration of the key / value / backoff parameters
While it'd be nice to have something like a `CryptoStoreLockGuard`, it's hard to implement without being racy, because of the `async` statements that would happen in the `Drop` method (and async drop isn't stable yet).
## Test program
There's also a test program in which I shamelessly show my rudimentary unix skills; I've put it in the `labs/` directory but this could as well be a large integration test. A parent program initially fills a custom crypto store, then creates a `pipe()` for 1-way communication with a child created with `fork()`; then the parent sends commands to the child. These commands consist in reading and writing into the crypto store, using a lock. And while the child attempts to perform these operations, the parent tries hard to get the lock at the same time. This helps figuring out a few issues and making sure that cross-process locking would work as intended.
This patch uses a new `async-once-cell` crate to make `Room::timeline`
and `Room::sneaky_timeline` by making them lazy. Their values are
computed lazily, on-demand, when calling the `Room::timeline` method or
`Room::latest_event`.
This was a temporary replacement I've made, waiting for a PR of mine to be merged, and it's been merged since then. Oh well.
Signed-off-by: Benjamin Bouvier <public@benj.me>
This patch adds a new method on `SlidingSyncList` named
`room_list_filtered_stream`, which uses the new `eyeball-im-util` crate
with its new `FilteredSubscriber` type.
This patch does several things.
First, `SlidingSync::on_list` is now async, and accept async closures.
Second, `SlidingSync::lists` and `::rooms` are behind an `AsyncRwLock`
instead of a `StdRwLock`. The rest of the patch updates the consequence
of this.
