This patch adds the new `from_all_chunks` function in the
`linked_chunk::lazy_loader` module. It is only used for testing
purposes. It aims at replacing `LinkedChunkBuilderTest` (see next
patches). Why? Because `from_all_chunks` uses `from_last_chunk` and
`insert_new_first_chunk`: if `from_all_chunks` is able to find all
errors that `LinkedChunkBuilderTest` finds, it's a bingo. Transitively,
it proves that `from_last_chunk` and `insert_new_first_chunk` are
correct!
This patch updates `Update::RemoveChunk` to emit `VectorDiff::Remove`.
Until now, `RemoveChunk` was expecting the chunk to be
empty, because it is how it is used so far. However, with
https://github.com/matrix-org/matrix-rust-sdk/pull/4694, it can change
rapidly.
The code before this patch was doing this:
- look if there's any prev-batch token available right now, aka look if
there's a gap in the in-memory linked chunk
- look at the first chunk; if it's a gap, return to the caller so it
resolves it
The check is done twice at two different levels, which is confusing.
Instead, this patch rewrites it so that the chunk is done only in
`load_more_event_backwards()`.
Note this is also correct for the case storage is disabled; in this
case, we early return and always try to resolve the gap anyways.
This patch renames the `builder` module to `lazy_loader`.
The `LinkedChunkBuilder`'s methods are now functions.
The `LinkedChunkBuilder` struct is removed. Finally,
`LinkedChunkBuilderError` is renamed `LazyLoaderError`.
The `LinkedChunkBuilderTest` struct is kept for the moment. It's going
to be replaced soon.
This patch adds an index on `events.event_id` and on `events.room_id`
so that queries on this column are faster. It mostly happens for the
`Deduplicator`, which runs for every backwards pagination or sync.
This patch also updates the query in `filter_duplicated_events` to
sort event by their `chunk_id` and `position` so that the results are
constant, it helps when testing.
This is the method to get the server metadata in the latest draft of
[MSC2965](https://github.com/matrix-org/matrix-spec-proposals/pull/2965).
We still keep the old behavior with `GET /auth_issuer` as fallback for
now because it has wider server support.
There are some pre-main commit cleanups to simplify the main commit.
This can be reviewed commit by commit.
The changes were tested with the oidc_cli example on beta.matrix.org.
Closes#4550.
---------
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch moves `chunk_debug_string` from `rooms/mod.rs` to
`rooms/event.rs`. In addition, it restores (and rewrites) a test,
initially for `chunk_debug_string`, now for `RoomEvents::debug_string`
whichh is the public API.
This patch changes the semantics regarding what to do in case of
duplicated events received during a backwards pagination.
Previously, the strategy for both sync and backwards pagination was
the same: With the new received events, when duplicated events are
detected, the old events are removed and the new ones are kept.
The strategy is reversed for backwards pagination: the old events are
always kept, and the duplicated events are removed from the new events.
The rest of the patch is about removing dead code because of this
change.
This patch updates `Pagination::run_backwards_impl` to paginate in the
event cache. The flow is now as follow:
- backwards pagination tries to load and to insert a new previous chunk
from the store
- if the new chunk contains events, they are returned, pagination done
- if the new chunk is a gap, the flow continues
- (as previously) check for a prev batch token (it exists in the newly
inserted gap)
- (as previously) run a network request, replace the gap by the new
events
- etc.
The new part is to load and to insert a new previous chunk. The rest
is stays the same. The code has been moved in code to keep the lock
releases happy and to clarify the code.
This patch adds the `RoomEventCacheState::load_more_events_backwards`
method to load a new chunk and to insert it at the beginning of the
`LinkedChunk`.
It uses the new `EventCacheStore::load_previous_chunk` method, along
with the new `LinkedChunkBuilder::insert_new_first_chunk` method.
This patch updates `RoomEventCacheState::new` to load a single chunks
of events instead of all events. It solves bugs where all events were
loaded, while removing the gaps in between, thus the `Timeline` wasn't
able to load the missing events to fill the gaps.
This patch removes a `TODO` in `BackPaginationOutcome`.
Events it contains are deduplicated by the `EventCache` (see
`event_cache::Deduplicator`) when inserted inside `RoomEventCache`.
This patch adds a test for all event cache store
implementations that tests a linked chunk incremental
loading, i.e. the `EventCacheStore::load_last_chunk` and
`EventCacheStore::load_previous_chunk` methods.
Most of the methods on `LinkedChunkBuilder` are now only used
for testing. This patch splits `LinkedChunkBuilder` and creates
`LinkedChunkBuilderTest`. This new type is part of the public
API because it's used in other crates, but it's hidden from the
documentation.
This patch adds two methods on `LinkedChunkBuilder`: `from_last_chunk`
to build a new `LinkedChunk` with a single provided chunk, and
`insert_new_first_chunk` to insert a new chunk at the beginning of the
provided `LinkedChunk`.
This patch renames `RoomEvents::with_initial_chunks` to
`with_initial_linked_chunk`. It avoids a confusion between several
chunks, like `RawChunk`s, and `LinkedChunk` which represents several
`Chunk`s.
This patch update the `EventCacheStore` trait to:
1. rename `reload_linked_chunk` into `load_all_chunks` and put this
method behind `#[cfg(test)]` so that it is removed from the public API,
2. add `load_last_chunk`,
3. add `load_previous_chunk`.
These 2 new methods are implemented inside the `MemoryStore` (with its
real implementation in the `RelationalLinkedChunk`), but `todo!()` are
added for the SQLite implementation.
The format changed 10 months ago and since it contains the tokens, it should have be reserialized already in that time.
Afaict EX clients do not serialize that type, the bindings have their own `Session` type for that.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
