Commit Graph

163 Commits

Author SHA1 Message Date
Valere
31e9600078 feat(send_queue): Persist failed to send errors (#4137)
Modify the SendQueue in order to persist the error that cause the event
to fail to send as a `QueueWedgeError`. The `QueueWedgeError` is not a
1:1 mapping for all kinds of errors, but holds variant and information
that the client can react to in order to propose "quick fixes"/solution
before retrying to send.

Fixes https://github.com/matrix-org/matrix-rust-sdk/issues/3973 
Also fixes https://github.com/element-hq/element-x-ios/issues/3287
because when a timeline reset occurs the fail to send reason is also
lost.

This PR starts with a refactoring commit
e7696003e8
to introduce the new `QueueWedgedError` and move the logic that was in
the ffi layer to convert api errors to SendState error variant. This
`QueueWedgedError` can be directly use in the `SendingFailed` variant
and expose to ffi.

Second commit
109c133746
adds the persistence, `QueuedEvent` now have an optional error field
instead of a `is_weged` boolean. Same for LocalEchoContent::Event.
Adds also Migration for sqlite and indexeddb

Co-authored-by: Benjamin Bouvier <benjamin@bouvier.cc>

Changelog:  We now persist the error that caused an event to fail to send. The error `QueueWedgeError` contains info that client can use to try to resolve the problem when the error is not automatically retry-able. Some breaking changes occurred in the FFI layer for `timeline::EventSendState`, `SendingFailed` now directly contains the wedge reason enum; use it in place of the removed variant of `EventSendState`.
2024-10-23 12:48:55 +02:00
Damir Jelić
1945b508c3 Create some missing changelog files 2024-10-10 14:32:46 +02:00
Damir Jelić
ca7f2ad3d0 Add a cargo-release config 2024-10-10 14:32:46 +02:00
Kévin Commaille
1e0e815fab base: Remove deprecated StateStore APIs
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-09-30 12:18:10 +02:00
Kévin Commaille
b0e8121347 sqlite: Bump sqlite crates
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-09-04 10:55:38 +02:00
Richard van der Hoff
7bcc920514 sqlite: add get_inbound_group_sessions_for_device_batch 2024-09-02 18:07:38 +01:00
Richard van der Hoff
12653fb2b6 sqlite: add new curve_key and sender_data_type columns 2024-09-02 18:07:38 +01:00
Richard van der Hoff
228a117ccb crypto: add get_inbound_group_sessions_for_device_batch to CryptoStore 2024-09-02 18:07:38 +01:00
Kévin Commaille
7f4e79e2a3 Add link to SQLite docs
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-09-02 17:44:30 +02:00
Kévin Commaille
7807ed8bda sqlite: Update last access time first to force write transaction
Avoids errors when the read transaction tries to upgrade to a write transaction.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-09-02 17:44:30 +02:00
Benjamin Bouvier
b8d90286aa testing: enforce a test_ prefix for tests
This will only apply to `async_test` functions, but I think this is a
win:

1. for consistency within the codebase, since I've started doing so in
many places,
2. because these function names will clearly identify these functions as
tests, in the call tree interfaces, when rendered using the LSP
show-callers/show-callees functionality.
2024-09-02 12:02:43 +02:00
Kévin Commaille
a8b38d271e sqlite: Merge init and run_migrations for SqliteEventCacheStore
This was copied from SqliteStateStore, but the reason
that they are separated there is because some migrations
require the store cipher.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-29 09:31:40 +03:00
Kévin Commaille
66e901bb9b sqlite: Make migrations atomic
Setting the version number only when all migrations are done
means that the version will be wrong if a migration fails.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-29 09:31:40 +03:00
Kévin Commaille
d17a49e827 sqlite: Add more methods to SqliteKeyValueStoreAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille
1321e92b30 sqlite: Rename SqliteObjectStoreExt to SqliteKeyValueStoreAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille
6e62f8b269 sqlite: Rename utils::ConnectionExt to SqliteKeyValueStoreConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille
6e369aecc9 sqlite: Rename SqliteConn to SqliteAsyncConn and SqliteObjectExt to SqliteAsyncConnExt
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-28 15:45:12 +02:00
Kévin Commaille
b9d49b85c3 sqlite: Use transaction for chunked queries
Allows the operation to be atomic.
Also allows to chunk part of a transaction.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 18:20:41 +02:00
Kévin Commaille
fe143ffbed sqlite: Remove SqliteObjectEventCacheStoreExt trait
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 10:40:12 +02:00
Kévin Commaille
25e406f669 sqlite: Remove unused path field
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 10:40:12 +02:00
Kévin Commaille
154f86aa20 sqlite: Rename deadpool_sqlite::Object to SqliteConn consistently
So we know that its always the same type when reading the code.

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-26 10:40:12 +02:00
Kévin Commaille
01e2db1f52 base: Move media cache to new EventCacheStore trait (#3858)
Allows to save media in a different path than the state store.

This adds a "last_access" field to the SQLite implementation, to prepare
for future work on a media retention policy.

This removes the IndexedDB media cache implementation, because as far as
I know it is currently unused, and I have no idea how to implement
efficiently the planned media retention policy with a key-value store.

Closes #1810.

- [x] Public API changes documented in changelogs (optional)

---------

Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
2024-08-22 10:36:43 +02:00
Andy Balaam
ca09917d84 crypto: Update the comment about source of truth for in sqlite 2024-08-20 15:36:34 +01:00
Andy Balaam
78924ed877 crypto: In sqlite, use the SQL column value for backed_up everywhere
Most times we pulled an InboundGroupSession from the sqlite DB, we were
overriding whatever value for `backed_up` was stored inside the pickled
value, and using the value stored in the SQL column.

But when we pulled a single InboundGroupSession from the DB by ID, we
did not override it.

I am fairly sure this was an accidental oversight, so this change
corrects it, and unifies the code with other places we create these
objects.
2024-08-20 15:36:34 +01:00
Andy Balaam
668a267c9b crypto: Merge deserialize_pickled_inbound_group_session into unpickle_inbound_group_session 2024-08-20 13:16:17 +01:00
Andy Balaam
fed08fad76 crypto: Utility deserialize_and_unpickle_inbound_group_session for sqlite store 2024-08-20 13:16:17 +01:00
Damir Jelić
96b615ba8e Remove Olm Session cache from the individual crypto store implementations 2024-08-06 15:10:13 +02:00
Benjamin Bouvier
ce68ad4968 state store: add a transparent newtype ChildTransactionId to help distinguish the parent from the child transaction id 2024-07-25 15:20:34 +02:00
Benjamin Bouvier
2eb6930988 send queue: add an own transaction id for dependent events
The previously named `transaction_id` is also renamed to
`parent_transaction_id` to make it clearer.
2024-07-24 17:54:25 +02:00
Benjamin Bouvier
9a7f18c62c state store: add dependent queued events tables and operations 2024-07-24 17:54:25 +02:00
Damir Jelić
d65e33ca6a Merge branch '0.7-release' into main 2024-07-19 12:02:40 +02:00
Damir Jelić
dda080c497 sqlite: Bump the version 2024-07-19 10:18:27 +02:00
Damir Jelić
a5dbfa66a7 encryption: Rename the ReadOnly user identity types 2024-07-12 18:06:34 +02:00
Damir Jelić
6f0d3b663b encryption: Rename ReadOnlyDevice to DeviceData
ReadOnlyDevice is not particularly useful as a description of why we
have two device types. This commit renames it into DeviceData, as this
struct is used to hold the device keys and additional local device data.

I'm not quite sure why it took me so long to come up with a better name.

Please forgive me past readers.
2024-07-12 18:06:34 +02:00
Damir Jelić
9d464eb908 tests: Add a snapshot of a SQLite database to perform regression tests
The test database was created using a slightly modified `oidc-cli`
example, to turn of the database encryption, on commit
d6dca91df86413b0cbf193a4be191835dd81862e
2024-07-12 18:06:34 +02:00
Benjamin Bouvier
c7d1a7db4f store: save/restore server capabilities 2024-07-09 12:37:33 +02:00
Ivan Enderlin
02dddb47c9 fix(base) Move StateChanges::room_info_notable_changes into a standalone BTreeMap.
This patch extracts `StateChanges::room_info_notable_changes` as a
variable that is passed to `BaseClient::apply_changes`.
2024-07-08 18:47:55 +02:00
Ivan Enderlin
ec7fa76240 feat(base): Revisit the roominfo_update API.
First off, this patch removes the
`RoomInfoNotableUpdate::trigger_room_list_update` field. It is replaced
by a `reasons: RoomInfoNotableUpdateReasons` 8-bit unsigned integer. It
addresses the following issues:

1. When a subscriber receives a `RoomInfoNotableUpdate`, they have no
   idea what has triggered this update.
2. In
   `matrix_sdk_base::sliding_sync::BaseClient::process_sliding_sync_e2ee`,
   we were triggering an update even if the latest event wasn't
   modified: it is a false-positive, it was a bug and a waste of
   resources. Now it's more refined, see the why below.

Second, this patch removes the second `trigger_room_list_update`
argument of `matrix_sdk_base::BaseClient::apply_changes`. This method
now knows where to find the reasons for the room info notable updates,
see next point.

Third, this patch adds a new
`matrix_sdk::StateChanges::room_info_notable_updates` field which is a
B-tree map between an `OwnedRoomId` and a
`RoomInfoNotableUpdateReasons`. The idea is that all places that receive
a `StateChanges` can also create a room info notable update with a
specific reason. This is a finer grained mechanism, and it avoids to
pass new arguments everywhere. It's cleaner.

Finally, it's easier than ever to add a new reason and to propagate it
to subscribers.
2024-07-08 18:47:55 +02:00
Richard van der Hoff
d60ec55e30 Crypto store: clear db before integ tests (#3644)
It's currently possible for integ test results to leak from one test run to the
next (for example, the indexeddb stores hang around in the browser), causing
bad test results.

Extend the test setup routine to clear out the store before the test starts.
2024-07-03 13:39:54 +01:00
Benjamin Bouvier
2f125e97ee send queue: make SendHandle::abort/update more precise
Using `SendHandle::abort()` after the event has been sent would look
like a successful abort of the event, while it's not the case; this
fixes this by having the state store backends return whether they've
touched an entry in the database.
2024-07-01 16:19:48 +02:00
Benjamin Bouvier
45c6efcc65 state store: add method to update the content of a send queue event
It also resets the wedged state, so that the queue will retry to send
this event later. This will show useful in the following case: when an
event is too big, we can now retry to send it, even if it was blocked,
by splitting the event instead of copy/abort/recreate it.
2024-06-27 18:53:00 +02:00
Hubert Chathi
d41af396cc Embed device keys in Olm-encrypted messages (#3517)
This patch implements MSC4147[1].

Signed-off-by: Hubert Chathi <hubertc@matrix.org>

[1]: https://github.com/matrix-org/matrix-spec-proposals/pull/4147
2024-06-27 12:18:52 +02:00
Benjamin Bouvier
29ac3e0dda fixup! state store: change schema for send_queue_events table 2024-06-26 19:42:14 +02:00
Benjamin Bouvier
62801f1a6c state store: change schema for send_queue_events table
It turns out that so as to be able to read the room ids, they need to be
*values*, not only *keys* (since keys are one-way hashed). This means we
need to duplicate the room_id field in indexeddb/sqlite, so each entry
contains both the room_id as a key (for queries) and as a value (to
return it).

Since there's no meaningful migration we can apply, the way to go is to
drop the pending events table and recreate it from the ground up. It is
assumed that no one has used the store on indexeddb; otherwise the
workaround would be to drop and recreate it.
2024-06-26 19:42:14 +02:00
Benjamin Bouvier
74b770f4d6 send queue: use state-store backed storage for remembering events to be sent 2024-06-24 13:56:10 +02:00
Mauro
d035eb1d90 feat (bindings): added APIs to save and load composer drafts (#3531)
This PR is the first piece of the breakdown of the following PR: https://github.com/matrix-org/matrix-rust-sdk/pull/3439 where only the core functionalities of the feature are implemented, without addressing the issue of editing and replying to events that have not yet been paginated.
2024-06-11 15:32:07 +00:00
Richard van der Hoff
1510f87fa0 UTD hook: stop re-reporting of UTDs on app relaunch (#3519)
Use a Bloom filter to keep track of which events we have already reported to the parent UTD hook.
We load the data from database on startup, and flush it out immediately on every update.
2024-06-10 18:56:35 +01:00
Benjamin Bouvier
5093af87b1 clippy: disallow useless asyncs (#3513)
Using async when not required will increase compile times, and propagate async-ness to the callers, transitively.

See also the [lint description](https://rust-lang.github.io/rust-clippy/master/#/unused_async).

Since we only had a few false positives, I've enabled it by default.
2024-06-05 17:13:10 +02:00
Jonas Platte
02185f7034 Configure unexpected-cfgs via Cargo.toml instead of build scripts 2024-06-04 23:34:38 +02:00
Richard van der Hoff
4cd619ccdd crypto: New method CryptoStore::save_inbound_group_sessions
When we add a batch of inbound group sessions to the store, if they came from a
backup, we need to record which backup version they came from.
`CryptoStore::save_changes` doesn't give us an easy way to set the backup
version (we could add it to the `Changes` struct, but ugh).

So, we add a new method `save_inbound_group_sessions` which takes the backup
version as a separate param. This works fine, because whenever we import
sessions there are no other changes to store.

The new method isn't used outside of tests yet -- that comes in a follow-up
commit.
2024-05-23 15:52:22 +01:00