mirror of
https://github.com/matrix-org/matrix-rust-sdk.git
synced 2026-05-24 16:48:52 -04:00
6f42210d6a
This patch changes the query used by `SqliteEventCacheStore::load_all_chunks_metadata`. It was the cause of severe slowness. The new query improves the throughput by +1140% and the time by -91.916%. The benchmark will follow in the next patch. Metrics for 10'000 events (with 1 gap every 80 events). - Before: - throughput: 20.686 Kelem/s, - time: 483.43 ms. - After: - throughput: 253.52 Kelem/s, - time: 39.478 ms. This query will visit all chunks of a linked chunk with ID `hashed_linked_chunk_id`. For each chunk, it collects its ID (`ChunkIdentifier`), previous chunk, next chunk, and number of events (`num_events`). If it's a gap, `num_events` is equal to 0, otherwise it counts the number of events in `event_chunks` where `event_chunks.chunk_id = linked_chunks.id`. Why not using a `(LEFT) JOIN` + `COUNT`? Because for gaps, the entire `event_chunks` will be traversed every time. It's extremely inefficient. To speed that up, we could use an `INDEX` but it will consume more storage space. Finally, traversing an `INDEX` boils down to traverse a B-tree, which is O(log n), whilst this `CASE` approach is O(1). This solution is nice trade-off and offers great performance.