Handle missing devices in /keys/claim responses#2805
Conversation
We can no longer randomly inject `/keys/claim` responses when there was no pending request; instead we have to generate a request using `get_missing_sessions`. *However*, the whole point is that Alice's device should be "timed out", so will not appear in the generated request. So, to test the timeout behaviour, we need to forcibly expire the timeout, and *then* generate a request; and once the valid response is processed, check that the timout is properly cleared.
... and make sure that any responses received match it.
Compare the response to `/keys/claim` with the request, so that we can see which devices are absent in the response, and handle accordingly.
83ab51a to
454a7b9
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #2805 +/- ##
==========================================
+ Coverage 82.01% 82.06% +0.04%
==========================================
Files 204 204
Lines 21021 21043 +22
==========================================
+ Hits 17240 17268 +28
+ Misses 3781 3775 -6
☔ View full report in Codecov by Sentry. |
poljar
left a comment
There was a problem hiding this comment.
Looks good, left a couple of suggestions.
| // empty map, this will yield `None`. | ||
| // | ||
| // The `None` entries are then filtered out by `filter_map`. | ||
| key_map.values().next().map(|key| ((user_id.clone(), device_id.clone()), key)) |
There was a problem hiding this comment.
| key_map.values().next().map(|key| ((user_id.clone(), device_id.clone()), key)) | |
| key_map.values().next().map(|key| ((user_id.to_owned(), device_id.to_owned()), key)) |
| // if we were able to pair this response with a request, look for devices that | ||
| // were present in the request but did not elicit a successful response. | ||
| if let Some(request) = request { | ||
| let missing_devices: Vec<(OwnedUserId, OwnedDeviceId)> = request |
There was a problem hiding this comment.
I think that this whole thing can be better expressed via sets, something like:
let sessions_to_create: BTreeSet<_> =
sessions_to_create.iter().map(|((u, d), _)| (u, d)).collect();
let requested_sessions: BTreeSet<(_, _)> = request
.one_time_keys
.iter()
.filter(|(user_id, _)| !failed_servers.contains(user_id.server_name()))
.flat_map(|(u, d)| d.keys().map(|d| (u, d)).collect::<BTreeSet<_>>())
.collect();
let missing_devices: BTreeSet<_> = requested_sessions.difference(&sessions_to_create).collect();*edit: Perhaps even move the filter() for the failed servers on the let missing_devices line.
You'll have to deal with double borrows a bit further bellow, but I think the clarity is worth it:
for (user_id, device_id) in missing_devices {
failed_devices_lock
.entry((*user_id).to_owned())
.or_default()
.insert((*device_id).to_owned())
}There was a problem hiding this comment.
ok, happy to do this. The reason I avoided it was that it felt a bit redundant to build the two sets just so that we could call .difference on them, particularly given we already have the requested_sessions in a map whose keys we can check in O(n log n) (ie, the same as the BTreeSet).
I guess using sets shows the intention more clearly though.
There was a problem hiding this comment.
Luckily we can check if the BTreeSet approach is slower, red is the flat_map(), blue is the BTreeSet approach:
The benchmark is processing a /keys/claim response with 163 one-time keys. I did run it a couple of times to get somewhat stable results. The BTreeSet approach does not seem to make any measurable difference.
There was a problem hiding this comment.
To check I am understanding the report correctly: this is saying the mean time increased by 0.7680%? In which case, very much agreed this is not worth worrying about.
I'd be interested to see how you ran this benchmark, for future reference.
There was a problem hiding this comment.
You are understanding correctly, I did have a couple of runs which claimed that the time decreased by ~0.5% as well. As the report tells us, all of this is within the noise threshold.
I'd be interested to see how you ran this benchmark, for future reference.
Please take a look at the /benchmarks folder: https://github.com/matrix-org/matrix-rust-sdk/tree/main/benchmarks
Co-authored-by: Damir Jelić <poljar@termina.org.uk>
/keys/claim
matrix-org/matrix-spec-proposals#4072

Keep a record of devices that were included in a
/keys/claimrequest, and then, if they are missing in the response, register them as "failed".Fixes #281.
This PR is currently based on #2803.