9f2c572709
## Some context
An aggregation is an event that relates to another event: for instance,
a
reaction, a poll response, and so on and so forth.
## Some requirements
Because of the sync mechanisms and federation, it can happen that a
related
event is received *before* receiving the event it relates to. Those
events
must be accounted for, stashed somewhere, and reapplied later, if/when
the
related-to event shows up.
In addition to that, a room's event cache can also decide to move events
around, in its own internal representation (likely because it ran into
some
duplicate events, or it managed to decrypt a previously UTD event).
When that happens, a timeline opened on the given room
will see a removal then re-insertion of the given event. If that event
was
the target of aggregations, then those aggregations must be re-applied
when
the given event is reinserted.
## Some solution
To satisfy both requirements, the [`Aggregations`] "manager" object
provided
by this PR will take care of memoizing aggregations, **for the entire
lifetime of the timeline** (or until it's clear'd by some
caller). Aggregations are saved in memory, and have the same lifetime as
that of a timeline. This makes it possible to apply pending aggregations
to cater for the first use case, and to never lose any aggregations in
the
second use case.
## Some points for the reviewer
- I think the most controversial point is that all aggregations are
memoized for the entire lifetime of the timeline. Would that become an
issue, we can get back to some incremental scheme, in the future:
instead of memoizing aggregations for the entire lifetime of the
timeline, we'd attach them to a single timeline item. When that item is
removed, we'd put the aggregations back into a "pending" stash of
aggregations. If the item is reinserted later, we could peek at the
pending stash of aggregations, remove any that's in there, and reapply
them to the reinserted event. This is what the [first version of this
patch](ec64b9e0bc)
did, in a much more adhoc way, for reactions only; based on the current
PR, we could do the same in a simpler manner
- while the PR has small commits, they don't quite make sense to review
individually, I'm afraid, as I was trying to find a way to make a
general system that would work not only for reactions, poll responses
and ends. As a matter of fact, the first commits may have introduced
code that is changed in subsequent commits, making the review a bit
hazardous. Happy to have a live reviewing party over Element Call, if
that helps, considering the size of the patch.
- future work may include using the aggregations manager for edits too,
leading to more code removal.
Matrix SDK integration test
This set of tests requires a Synapse instance, and it runs the tests from this directory against this real-world server. As a result, these tests depend on the load of the machine/server, and as such they might be more sensitive to timing issues than other tests in the code base.
Requirements
This requires a synapse backend with a configuration patched for CI. You can get it up and running
with docker compose via:
docker compose -f assets/docker-compose.yml up -d
docker compose -f assets/docker-compose.yml logs --tail 100 -f
Note that this works also with podman compose.
Patches
You can see the patches we do to configuration (namely activate registration and resetting rate
limits, enable a few experimental features), check out what assets/ci-start.sh changes.
Running
The integration tests can be run with cargo test or cargo nextest run.
The integration tests expect the environment variables HOMESERVER_URL to be the HTTP URL to
access the synapse server and HOMESERVER_DOMAIN to be set to the domain configured in
that server. These variables are set to a default value that matches the default docker-compose.yml
template; if you haven't touched it, you don't need to manually set those environment variables.
Maintenance
Delete all instance data
To stop the services and drop the database of your docker-compose cluster, run:
docker compose -f assets/docker-compose.yml down --volumes --remove-orphan -t 0
Rebuild the synapse image
If the Synapse image has been updated in version, you may need to rebuild the custom Synapse image using the following:
docker compose -f assets/docker-compose.yml build --pull synapse
Then restart the synapse service:
docker compose -f assets/docker-compose.yml up -d synapse